read -t0 may report that input is available where none is possible
true | { sleep 1 ; read -rt0 ; echo $?; }
outputs 0. There can clearly be no data on the stdin of the second process in
the pipeline.
bash(1) has this to say about read -t:
"If timeout is 0, read returns immediately, without trying to read any data.
The exit status is 0 if input is available on the specified file descriptor,
non-zero otherwise."
Either bash considers EOF on the pipe "input", or this is a bug.
This was discussed 2 years ago
https://lists.gnu.org/archive/html/bug-bash/2019-12/msg00076.html. Chet's
conclusion then was that the detection of the presence of data on an fd is
inherently racy. That is true, but still read -rt0 should probably not report
"input is available" where there can be none. If the current behavior is
intentional, consider making it clear in the man page.
The implementation of input_avail() in Bash 5.x appears to just call select()
on the fd, but select() reports an fd readable also on EOF. The relevant quote
from man select(2):
"The file descriptors listed in readfds will be watched to see if characters
become available for reading (more precisely, to see if a read will not block;
in particular, a file descriptor is also ready on end-of-file)."
-Mark
Re: Strange complete -C output
Emily Seville wrote on Sun, Nov 21, 2021 at 01:40:08PM +1000: > Bash version: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu) > OS version: Linux Mint 20.2 Cinnamon > > Hello! Why when I create *complete -C 'echo -e "aa\nab\n"' : *completion and > try use it I obtain additional garbage in output instead of only *ab ab* > hints? Doesn't sound like garbage to me - it's the arguments the shell passes to your command, echo will just repeat them. If you replace echo -e with printf for example, that ignores extra arguments, you'll get what you want -- but if your only requirement is a fixed list then you're better off with something else than running a command, why not complete -W 'aa ab' ? This (bug-bash) mailing list is intended for actual bug reports, not usage questions. If you have questions in future, the best place to ask them is another mailing list run by this community: https://lists.gnu.org/mailman/listinfo/help-bash -- Asmadeus | Dominique Martinet
Strange complete -C output
Bash version: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu) OS version: Linux Mint 20.2 Cinnamon Hello! Why when I create *complete -C 'echo -e "aa\nab\n"' : *completion and try use it I obtain additional garbage in output instead of only *ab ab* hints?
Re: Unclosed quotes on heredoc mode
Date:Sat, 20 Nov 2021 15:19:33 -0500
From:Chet Ramey
Message-ID:
| Right. Purposeful.
There's a difference between done intentionally for pragmatic reasons,
and done intentionally because it is the right thing to do and people
should depend upon it remaining that way.
| How about this. You show me examples where bash (devel bash) does what you
| think is the wrong thing, and we agree it's a bug, I'll fix it.
I'll run our tests against the newest (released) bash (5.1.12(1)-release)
[what does the (1) represent?? It always seems to be (1) in versions I see.]
| The devel bash already does this.
What the devel one does is unknown to me, I don't think I even have
the means to obtain it (I have nothing at all git related, and no interest
in changing that state of affairs).
| > and a newline token in the middle of
| > a command substitution counts for a here doc operator that occurred before
| > it,
|
| What does `counts' mean? You're not really reading the lines as shell
| words,
"counts" means "is the one that matters" (ie: do not ignore this one).
But, no, not this...
| cat << EOF
| echo $(echo this EOF is
| not the end of
| the command substitution
| EOF
| but it is the end of the
| here-document
| )
though that is a mildly interesting case, and I agree on how that
gets parsed (the contents of the here doc are not examined until it
is expanded when used for a redirection). That should result in a
redirection error for cat, then (probably) "but: not found" (if the
shell didn't already exit), "here-document: not found" and a syntax
error on the ')'. (The "not found" errors are, naturally, assuming
that commands of those names aren't found in a PATH search).
What I meant was this one:
cat From the line numbers, I assume the first is when scanning the outer cat
command, and detecting its cmdsub arg, and the 2nd is from rescanning the
command substitution. The first one clearly knows there is a heredoc,
it also knows it is yet to encounter a newline token (or any newline in
this example) hence the heredoc data cannot possibly be expected yet, it
must wait until after that newline - eventually it gets past >/dev/null,
finds the newline (token), and should start reading the heredoc text.
At that point it looks to see where the << redirection occurred (the first on
the line since this is the first heredoc read) and associates the data
with that redirection operator. When the cmdsub is ready to be executed
it finds the heredoc data already read and available.
I never got to enter the lines starting "abc" ... (I could have, but I know
I would have just seen 3 command not found errors, one for each line, so I
didn't bother.)
In both of those, the first newline token following the << operator (and its
word) is the one at the end of the first line (of each). The heredoc data
for each therefore starts on the 2nd line.
What should happen:
[jinx]{3}$ cat < foobar
> EOF
> echo barfoo) *.c
foobar
[jinx]{3}$ cat $( cat abc
> def
> FILES
cat: abc: No such file or directory
cat: def: No such file or directory
For the first there are a couple of .c files in $PWD but they don't contain
"barfoo", Neither "abc" nor "def" exist in $PWD
| > and a here doc operator in a command substitution might not encounter
| > a newline until after the cmdsub text has ended - the next following
newline
| > token provides there here doc text.
|
| I can't imagine a useful example of this that isn't an error.
That's the 2nd example above, and a very normal thing to want to do, very
short command substitutions (most of them) prefer to be complete within 1 line.
Note that neither in POSIX, nor anywhere else, has there ever been any
requirement on the heredoc data other than that it comes after the next
newline (which should, we agree, be newline token, not newline character).
Since heredocs are a lexical object, this processing is totally unaffected
by whatever semantics the grammar is extracting from the tokens the lexer is
returning to it, the grammar just increments the "number of heredocs needed"
counter, supplies the end words for each, and the lexer takes care of the rest.
And then there is of course the combination of the two of those examples:
cat redirect is moved before the cmdsub).
Re: Unclosed quotes on heredoc mode
On 11/20/21 12:35 PM, Robert Elz wrote: Date:Sat, 20 Nov 2021 11:33:37 -0500 From:Chet Ramey Message-ID: <4addb789-50b6-12a5-7b8a-8a082abaa...@case.edu> | I'm skeptical, but willing to be convinced. Bourne's shell allowed EOF to | terminate all sorts of things (quoted strings, command substitutions, here | documents) -- enough to make it purposeful. More likely economical. Making things fit in that sh was a real challenge. Right. Purposeful. That's a good starting point, provided you're willing to actually implement that. That's what I'd like. How about this. You show me examples where bash (devel bash) does what you think is the wrong thing, and we agree it's a bug, I'll fix it. But for this you need to understand that the shell has to parse and understand command substitutions, as they're read, in order to correctly find the end, The devel bash already does this. We've talked about it before. You need to use bison, not byacc, and a new enough version of bison, but it works fine. and a newline token in the middle of a command substitution counts for a here doc operator that occurred before it, What does `counts' mean? You're not really reading the lines as shell words, so a command substitution isn't really a command substitution while you're reading the body of a here-document. You mean something like this? cat << EOF echo $(echo this EOF is not the end of the command substitution EOF but it is the end of the here-document ) and a here doc operator in a command substitution might not encounter a newline until after the cmdsub text has ended - the next following newline token provides there here doc text. I can't imagine a useful example of this that isn't an error. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Unclosed quotes on heredoc mode
Date:Sat, 20 Nov 2021 11:33:37 -0500 From:Chet Ramey Message-ID: <4addb789-50b6-12a5-7b8a-8a082abaa...@case.edu> | I'm skeptical, but willing to be convinced. Bourne's shell allowed EOF to | terminate all sorts of things (quoted strings, command substitutions, here | documents) -- enough to make it purposeful. More likely economical. Making things fit in that sh was a real challenge. | > So just how many complaints do you get about the warning message? | > "ksh doesn't complain wbout this, why does bash?" | It's usually people who have misplaced or mistyped the ending delimiter. Yes, exactly - it happens like that all the time. That's why it should be an error, not just a warning - no different than when someone does cmd >/tpm/foo they made a typo, they get an error, and fix it. No problems. | "When an io_here token has been recognized by the grammar (see Shell | Grammar), one or more of the subsequent lines immediately following the | next NEWLINE token form the body of one or more here-documents and shall be | parsed according to the rules of Here-Document." That's a good starting point, provided you're willing to actually implement that. That's what I'd like. But for this you need to understand that the shell has to parse and understand command substitutions, as they're read, in order to correctly find the end, and a newline token in the middle of a command substitution counts for a here doc operator that occurred before it, and a here doc operator in a command substitution might not encounter a newline until after the cmdsub text has ended - the next following newline token provides there here doc text. | That implies that the shell goes off and | reads lines before parsing the rest of the current line as a list. Yes, certainly - how to read the here doc seems to be agreed, just when to read it is not. | > cat < file ; echo "abc | > def | > EOF | > ghi" \ | > EOF | > EOF | > What is the here doc, and what does echo say. | | That's a good example. The here-doc is empty (the delimiter is the third | EOF) and the echo prints the rest of the text, with the backslash-newline | disappearing. I agree. | I'd say that this is somewhat deceptive, and is a decent illustration of my | point. The shell -- bash, at least -- always reads complete lines from the | input before parsing any here documents, so it's going to keep reading | through the second EOF to read the `complete' first line, due to the quoted | string and the quoted newline. The `current' token is going to be the | newline that follows the second EOF even before it starts figuring out that | it has a here-document and goes off to collect the body. That's reasonable. As long as you stick to reading lines, and parsing them as they're read, and then insert here doc contents as soon as a here doc operator is located on one of the lines read. | > For this. No. An extension. One that comes for feee. | | I like the Freudian slip there. Oops... didn't spot that one! kre
Re: Unclosed quotes on heredoc mode
On 11/19/21 9:18 AM, Robert Elz wrote: illusory compat issues. I have no idea what inspired this initially, but my guess would be a code bug no-one noticed. I'm skeptical, but willing to be convinced. Bourne's shell allowed EOF to terminate all sorts of things (quoted strings, command substitutions, here documents) -- enough to make it purposeful. So just how many complaints do you get about the warning message? "ksh doesn't complain wbout this, why does bash?" It's usually people who have misplaced or mistyped the ending delimiter. It took only a few seconds to find this: https://unix.stackexchange.com/questions/657488/warning-here-document-at-line-2-delimited-by-end-of-file-wanted-eof I don't have time right now to look for other reports that might have tested it against other shells. | Which instance of `ola"'? The first or the second? The first. | This cannot be a serious question unless you mean the second. It is a very serious question, but not as to what should hppen but how the standard needs to describe it. That's why I suggested what I did. Some variant of the existing "When an io_here token has been recognized by the grammar (see Shell Grammar), one or more of the subsequent lines immediately following the next NEWLINE token form the body of one or more here-documents and shall be parsed according to the rules of Here-Document." could probably work as a basis. That implies that the shell goes off and reads lines before parsing the rest of the current line as a list. | The delimiter is a `word', and we both know what a shell word is. yes, but that's irrelevant, it is merely a coincidence here that the newline in question occurs in the delimiter. Another example cat < file ; echo "abc def EOF ghi" \ EOF EOF What is the here doc, and what does echo say. That's a good example. The here-doc is empty (the delimiter is the third EOF) and the echo prints the rest of the text, with the backslash-newline disappearing. I'd say that this is somewhat deceptive, and is a decent illustration of my point. The shell -- bash, at least -- always reads complete lines from the input before parsing any here documents, so it's going to keep reading through the second EOF to read the `complete' first line, due to the quoted string and the quoted newline. The `current' token is going to be the newline that follows the second EOF even before it starts figuring out that it has a here-document and goes off to collect the body. So, the shell reads the here-document body and creates the here document after it reads an unquoted newline token -- the first newline token after finding the here-document delimiter. The first newline after the << is the one after abc. Do remember that here doc data collection is entirely a lexical issue, that's why tgey dot appear anywhere in the sh grammar. Oh, I do. | The newline after the delimiter is both, but sure, newline token would | probably work better. The example above shows the issue better. That includes the \newline which can only be a \ newline because the 2nd char there is a newline, and that has to be seen at the lexical level. Yes. Here-documents are one of those features that requires mutual feedback between the parser and lexer. | So it doesn't read `lines' in the POSIX sense? Huh. Who knew? For this. No. An extension. One that comes for feee. I like the Freudian slip there. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
