Re: Not so useless use of cat
In article mailman.9035.1410965700.1147.bug-b...@gnu.org, Chet Ramey chet.ra...@case.edu wrote: On 9/17/14, 3:07 AM, Aharon Robbins wrote: I've considered emulating it everywhere, regardless of what the OS provides, but I'd get just as many complaints if I did that. Chet This is what gawk does. I haven't had any complaints about this, and once you do it that way you can claim that Bash is being consistent across all systems. (That's one of the reasons I did it that way.) My two cents, of course. Sure. It's a choice between internal and external consistency. If I emulated /dev/std* (and maybe /dev/fd/*) internally in bash, bash would behave the same everywhere, but, as Andreas said, I'd get questions about why `foo -o /dev/stdout' and `foo /dev/stdout' behaved differently. Yes, I live with the damned-if-you-do, damned-if-you-don't all the time too. It sounds like you've already decided which way you want things to be. :-) -- Aharon (Arnold) Robbins arnold AT skeeve DOT com P.O. Box 354Home Phone: +972 8 979-0381 Nof Ayalon D.N. Shimshon 9978500 ISRAEL
Re: Not so useless use of cat
On 9/18/14, 4:29 AM, Aharon Robbins wrote: Sure. It's a choice between internal and external consistency. If I emulated /dev/std* (and maybe /dev/fd/*) internally in bash, bash would behave the same everywhere, but, as Andreas said, I'd get questions about why `foo -o /dev/stdout' and `foo /dev/stdout' behaved differently. Yes, I live with the damned-if-you-do, damned-if-you-don't all the time too. It sounds like you've already decided which way you want things to be. :-) Yes, on this one. There are others; recall the `discussion' about whether bash should choose between /dev/fd or FIFOs for process substitution at runtime. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Not so useless use of cat
Chet Ramey chet.ra...@case.edu wrote: Yes, on this one. There are others; recall the `discussion' about whether bash should choose between /dev/fd or FIFOs for process substitution at runtime. That's a tougher one. It's a question of how far back do you wish to continue supporting systems? I'm finding that modern systems have pretty much the union of things that I need, and also that the older ones that don't simply aren't in use anymore. E.g., do you still need to support SunOS 4.1.x? Ultrix? OSF/1? Irix? I removed a lot of crufty code in gawk ~3 years ago to zero complaints. You might be able to do that in Bash (e.g., assume /dev/fd), especially if you do it a major release point (Bash 5.0). Again, my two cents' worth. And I understand that gawk doesn't make the same demands of the OS that Bash does. HTH, Arnold
Re: Not so useless use of cat
On Thu, Sep 18, 2014 at 07:26:33AM -0600, arn...@skeeve.com wrote: I'm finding that modern systems have pretty much the union of things that I need, and also that the older ones that don't simply aren't in use anymore. E.g., do you still need to support SunOS 4.1.x? Ultrix? OSF/1? Irix? We still have a couple Irix systems here. I removed a lot of crufty code in gawk ~3 years ago to zero complaints. You might be able to do that in Bash (e.g., assume /dev/fd), especially if you do it a major release point (Bash 5.0). Please don't do that! imadev:~$ ls -ld /dev/fd /dev/fd not found imadev:~$ uname -a HP-UX imadev B.10.20 A 9000/785 2008897791 two-user license
Re: Not so useless use of cat
On 9/18/14, 9:26 AM, arn...@skeeve.com wrote: Chet Ramey chet.ra...@case.edu wrote: Yes, on this one. There are others; recall the `discussion' about whether bash should choose between /dev/fd or FIFOs for process substitution at runtime. That's a tougher one. It's a question of how far back do you wish to continue supporting systems? I'm finding that modern systems have pretty much the union of things that I need, and also that the older ones that don't simply aren't in use anymore. E.g., do you still need to support SunOS 4.1.x? Ultrix? OSF/1? Irix? I removed a lot of crufty code in gawk ~3 years ago to zero complaints. You might be able to do that in Bash (e.g., assume /dev/fd), especially if you do it a major release point (Bash 5.0). The problem isn't old, unused code. It's the fact that some of the things bash wants to use are optional OS features. If bash is built on a machine, say FreeBSD, with /dev/fd available at configure time, that binary will attempt to use /dev/fd forever. This isn't such a problem when you're building bash yourself from ports, but it does inconvenience binary distributions that provide a lot of user-modifiable configuration options. The same build/distribution problem exists on Linux. The specific issue in the /dev/fd-FIFO case is using a feature like /dev/fd at a point where /dev/fd is not available (e.g., during system boot). Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Not so useless use of cat
In article mailman.8994.1410897983.1147.bug-b...@gnu.org,
Chet Ramey chet.ra...@case.edu wrote:
On 9/16/14, 3:00 PM, Bob Proulx wrote:
That is one of the reasons I don't like the /dev/std{err,in,out}
things. They are not portable. They do different things on different
systems. I avoid them.
I've considered emulating it everywhere, regardless of what the OS
provides, but I'd get just as many complaints if I did that.
Chet
This is what gawk does. I haven't had any complaints about this,
and once you do it that way you can claim that Bash is being consistent
across all systems. (That's one of the reasons I did it that way.)
My two cents, of course.
Arnold
--
Aharon (Arnold) Robbins arnold AT skeeve DOT com
P.O. Box 354Home Phone: +972 8 979-0381
Nof Ayalon
D.N. Shimshon 9978500 ISRAEL
Re: Not so useless use of cat
arn...@skeeve.com (Aharon Robbins) writes:
In article mailman.8994.1410897983.1147.bug-b...@gnu.org,
Chet Ramey chet.ra...@case.edu wrote:
On 9/16/14, 3:00 PM, Bob Proulx wrote:
That is one of the reasons I don't like the /dev/std{err,in,out}
things. They are not portable. They do different things on different
systems. I avoid them.
I've considered emulating it everywhere, regardless of what the OS
provides, but I'd get just as many complaints if I did that.
Chet
This is what gawk does. I haven't had any complaints about this,
and once you do it that way you can claim that Bash is being consistent
across all systems. (That's one of the reasons I did it that way.)
AWK can do that since it has full control over the files it accesses.
Bash doesn't have that, it needs to coexist with programs that open the
files themselves. It cannot emulate foo -o /dev/stdin to make it the
same as foo /dev/stdin.
Andreas.
--
Andreas Schwab, SUSE Labs, sch...@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
And now for something completely different.
Re: Not so useless use of cat
Am Tue, 16 Sep 2014 14:44:05 -0500 schrieb Dennis Williamson dennistwilliam...@gmail.com: Does your program support using a hyphen to represent stdout (some do)? program -i $i -o - It indeed does! Thanks for the tip. Ralf
Re: Not so useless use of cat
On 9/17/14, 3:07 AM, Aharon Robbins wrote: I've considered emulating it everywhere, regardless of what the OS provides, but I'd get just as many complaints if I did that. Chet This is what gawk does. I haven't had any complaints about this, and once you do it that way you can claim that Bash is being consistent across all systems. (That's one of the reasons I did it that way.) My two cents, of course. Sure. It's a choice between internal and external consistency. If I emulated /dev/std* (and maybe /dev/fd/*) internally in bash, bash would behave the same everywhere, but, as Andreas said, I'd get questions about why `foo -o /dev/stdout' and `foo /dev/stdout' behaved differently. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Not so useless use of cat
Am Sat, 13 Sep 2014 12:53:48 -0600
schrieb Bob Proulx b...@proulx.com:
Dennis Williamson wrote:
Bob Proulx wrote:
{ for i in file[12] ; do cat $i ; done ;} both
There's no need for the curly braces and the last semicolon.
Of course you are totally right. I was distracted by the subshell as
a concept. For re-stitching file with redirections a subshell isn't
needed and a list is convenient.
Note that the loop in this case can be replaced by
cat file[12] both
I failed to spot that in my earlier reply.
Me too. (But usually these are from more complex examples that can't
be simplified as much. It is just that all of the details aren't
shown.)
Actually things are more complicated. I do need the /dev/stdout part. I
obiously don't have the problem with `cat' but with some other program
that doesn't write to stdout per se and expects a -o parameter for the
output file. And this program just accepts one input file. I merely used
the first `cat' in my example to make my point. So what I wanted was
$ for i in file[12] ; do program -i $i -o /dev/stdout ; done outfile
which I assumed to be elegant and would do as I expected except it
didn't and I really thought it could have been a bug. That's why I
reported it here.
Thanks for all replys,
Ralf
Re: Not so useless use of cat
Ralf Goertz wrote: Actually things are more complicated. I do need the /dev/stdout part. I obiously don't have the problem with `cat' but with some other program that doesn't write to stdout per se and expects a -o parameter for the output file. And this program just accepts one input file. I merely used the first `cat' in my example to make my point. So what I wanted was I suspected that was the case. That is generally the way it goes. But does the program really have no way to write to stdout? I despise programs such as those. They should be fixed so that there is a way to use them as a filter. Because working around those programs by using Linux specific features such as /dev/stdout makes the entire bundle non-portable to systems that don't have /dev/stdout. $ for i in file[12] ; do program -i $i -o /dev/stdout ; done outfile which I assumed to be elegant and would do as I expected except it didn't and I really thought it could have been a bug. That's why I reported it here. I guess you could create as many temporary files as needed and then concatenate them with cat afterward. Or if the program can't be fixed to append or write to stdout directly then use the not so useless cat after all. Probably needs a comment in that case though! :-) There is another possibility. I don't actually like this one as much. It seems more confusing. It uses more processes. But try this: for i in file[12] ; do program -i $i -o (cat) ; done outfile Good luck! Bob
Re: Not so useless use of cat
On Tue, Sep 16, 2014 at 09:03:20AM +0200, Ralf Goertz wrote: Actually things are more complicated. I do need the /dev/stdout part. I obiously don't have the problem with `cat' but with some other program $ for i in file[12] ; do program -i $i -o /dev/stdout ; done outfile It's important to note that the following two cases are *not* equivalent: cat $i /dev/stdout program -i $i -o /dev/stdout In the first case, the /dev/stdout is part of a redirection. On platforms that do not have a native /dev/stdout in the file system, Bash handles this *internally* (and it would actually work the way you expected). In the second case, /dev/stdout is just a string as far as Bash is concerned. It is passed verbatim to program as an argument. There is no internal Bash magic happening. You get the underlying operating system implementation, or you get no such file or directory. You can try the process substitution that was suggested earlier, or you can explicitly make a named pipe and set up a background reader process.
Re: Not so useless use of cat
Greg Wooledge wrote:
It's important to note that the following two cases are *not*
equivalent:
cat $i /dev/stdout
program -i $i -o /dev/stdout
In the first case, the /dev/stdout is part of a redirection. On
platforms that do not have a native /dev/stdout in the file system,
Bash handles this *internally* (and it would actually work the way
you expected).
In the second case, /dev/stdout is just a string as far as Bash is
concerned. It is passed verbatim to program as an argument. There is
no internal Bash magic happening. You get the underlying operating
system implementation, or you get no such file or directory.
That is one of the reasons I don't like the /dev/std{err,in,out}
things. They are not portable. They do different things on different
systems. I avoid them.
Bob
Re: Not so useless use of cat
On Tue, Sep 16, 2014 at 2:03 AM, Ralf Goertz me@myprovider.invalid wrote:
Am Sat, 13 Sep 2014 12:53:48 -0600
schrieb Bob Proulx b...@proulx.com:
Dennis Williamson wrote:
Bob Proulx wrote:
{ for i in file[12] ; do cat $i ; done ;} both
There's no need for the curly braces and the last semicolon.
Of course you are totally right. I was distracted by the subshell as
a concept. For re-stitching file with redirections a subshell isn't
needed and a list is convenient.
Note that the loop in this case can be replaced by
cat file[12] both
I failed to spot that in my earlier reply.
Me too. (But usually these are from more complex examples that can't
be simplified as much. It is just that all of the details aren't
shown.)
Actually things are more complicated. I do need the /dev/stdout part. I
obiously don't have the problem with `cat' but with some other program
that doesn't write to stdout per se and expects a -o parameter for the
output file. And this program just accepts one input file. I merely used
the first `cat' in my example to make my point. So what I wanted was
$ for i in file[12] ; do program -i $i -o /dev/stdout ; done outfile
which I assumed to be elegant and would do as I expected except it
didn't and I really thought it could have been a bug. That's why I
reported it here.
Thanks for all replys,
Ralf
Does your program support using a hyphen to represent stdout (some do)?
program -i $i -o -
--
Visit serverfault.com to get your system administration questions answered.
Re: Not so useless use of cat
On 9/16/14, 3:00 PM, Bob Proulx wrote:
That is one of the reasons I don't like the /dev/std{err,in,out}
things. They are not portable. They do different things on different
systems. I avoid them.
I've considered emulating it everywhere, regardless of what the OS
provides, but I'd get just as many complaints if I did that.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Not so useless use of cat
Chet Ramey wrote:
Bob Proulx wrote:
That is one of the reasons I don't like the /dev/std{err,in,out}
things. They are not portable. They do different things on different
systems. I avoid them.
I've considered emulating it everywhere, regardless of what the OS
provides, but I'd get just as many complaints if I did that.
Agreed. There isn't any way to win.
Tonight I came up with yet another reason I don't like use of the
/dev/stdout and others. And also the use of /proc too. That reason
is use of programs in chroots. I ran into a program installer that
required that both /proc and /dev be mounted in the chroot I was
using. I can't see any reason those are actually needed. But it
required them both in order to function. Things like that get in the
way of using lightweight chroots which would otherwise work fine as a
light weight container.
Bob
Re: Not so useless use of cat
On Fri, Sep 12, 2014 at 06:12:13PM -0600, Bob Proulx wrote: (for i in file[12] ; do cat $i /dev/stdout ; done) both $ cat both second Because the /dev/stdout truncates the output. It writes the first. Then the second one truncates the file and then writes the second. The result is platform-specific, actually. I filed a bug report on this a few weeks ago, but Chet said it was working as expected. imadev:~$ uname -a HP-UX imadev B.10.20 A 9000/785 2008897791 two-user license imadev:~$ echo first file1; echo second file2 imadev:~$ for i in file[12]; do cat $i /dev/stdout; done both imadev:~$ cat both first second On HP-UX, /dev/stdout is NOT an actual file in the file system, so it is implemented by duplicating FD 1 within Bash. arc3:~$ uname -a Linux arc3 3.2.0-4-686-pae #1 SMP Debian 3.2.60-1+deb7u1 i686 GNU/Linux arc3:~$ for i in file[12]; do cat $i /dev/stdout; done both arc3:~$ cat both second On Linux, /dev/stdout is part of the file system, so Bash opens it and lets the operating system do what it will.
Re: Not so useless use of cat
Dennis Williamson wrote:
Bob Proulx wrote:
And the subshell isn't needed either. Use a list.
{ for i in file[12] ; do cat $i ; done ;} both
There's no need for the curly braces and the last semicolon.
Of course you are totally right. I was distracted by the subshell as
a concept. For re-stitching file with redirections a subshell isn't
needed and a list is convenient.
Note that the loop in this case can be replaced by
cat file[12] both
I failed to spot that in my earlier reply.
Me too. (But usually these are from more complex examples that can't
be simplified as much. It is just that all of the details aren't shown.)
Bob
Re: Not so useless use of cat
On Sep 12, 2014 6:42 PM, Ralf Goertz me@myprovider.invalid wrote: Hi, Why do I need cat (the second on) here? $ echo first file1 $ echo second file2 $ (for i in file[12] ; do cat $i /dev/stdout ; done) | cat both $ cat both first second If I omit the | cat after the loop I get $ cat both second Even when using both instead of both only second makes it into both. Why? I would have thought that using a subshell should be enough to protect the both from being overwritten. Ralf Also remove the redirection to stdout and the subshell for ... done both
Re: Not so useless use of cat
Ralf Goertz me@myprovider.invalid wrote:
Since you have used an invalid address I assume you are reading the
mailing list via a web archive or other means and did not CC you.
Why do I need cat (the second on) here?
You don't.
$ echo first file1
$ echo second file2
$ (for i in file[12] ; do cat $i /dev/stdout ; done) | cat both
Why do you want /dev/stdout there?
$ cat both
first
second
Okay.
If I omit the | cat after the loop I get
(for i in file[12] ; do cat $i /dev/stdout ; done) both
$ cat both
second
Because the /dev/stdout truncates the output. It writes the first.
Then the second one truncates the file and then writes the second.
Remove that from the line and it will print both lines.
(for i in file[12] ; do cat $i ; done) both
Even when using both instead of both only second makes it
into both. Why?
All that the , , 21, types of redirections do is to attach the
associated files with different files. Once associated the further
actions such as truncation happen to those files. Therefore the
truncation associated with /dev/null truncates the file that on the
far right was both.
Try this:
(for i in file[12] ; do cat $i /dev/stdout ; done) both
But really I think you are better off forgetting about /dev/stdout.
That is almost like the useless use of cat.
And the subshell isn't needed either. Use a list.
{ for i in file[12] ; do cat $i ; done ;} both
I would have thought that using a subshell should be
enough to protect the both from being overwritten.
The subshell isn't really part of the issue. The issue is that the
file descriptor is attached and truncated and that is orthogonal to
process boundaries.
Hope that helps!
In the future please consider asking questions such as these on
help-b...@gnu.org for general discussion of bash shell scripting. If
you expose a bug then report the bug to the bug reporting address.
The help-bash list is for help with bash scripting and this would have
been perfect there.
Bob
Re: Not so useless use of cat
On Sep 12, 2014 7:12 PM, Bob Proulx b...@proulx.com wrote:
Ralf Goertz me@myprovider.invalid wrote:
Since you have used an invalid address I assume you are reading the
mailing list via a web archive or other means and did not CC you.
Why do I need cat (the second on) here?
You don't.
$ echo first file1
$ echo second file2
$ (for i in file[12] ; do cat $i /dev/stdout ; done) | cat both
Why do you want /dev/stdout there?
$ cat both
first
second
Okay.
If I omit the | cat after the loop I get
(for i in file[12] ; do cat $i /dev/stdout ; done) both
$ cat both
second
Because the /dev/stdout truncates the output. It writes the first.
Then the second one truncates the file and then writes the second.
Remove that from the line and it will print both lines.
(for i in file[12] ; do cat $i ; done) both
Even when using both instead of both only second makes it
into both. Why?
All that the , , 21, types of redirections do is to attach the
associated files with different files. Once associated the further
actions such as truncation happen to those files. Therefore the
truncation associated with /dev/null truncates the file that on the
far right was both.
Try this:
(for i in file[12] ; do cat $i /dev/stdout ; done) both
But really I think you are better off forgetting about /dev/stdout.
That is almost like the useless use of cat.
And the subshell isn't needed either. Use a list.
{ for i in file[12] ; do cat $i ; done ;} both
I would have thought that using a subshell should be
enough to protect the both from being overwritten.
The subshell isn't really part of the issue. The issue is that the
file descriptor is attached and truncated and that is orthogonal to
process boundaries.
Hope that helps!
In the future please consider asking questions such as these on
help-b...@gnu.org for general discussion of bash shell scripting. If
you expose a bug then report the bug to the bug reporting address.
The help-bash list is for help with bash scripting and this would have
been perfect there.
Bob
There's no need for the curly braces and the last semicolon.
Note that the loop in this case can be replaced by
cat file[12] both
I failed to spot that in my earlier reply.
