Re: printf -u "$fd"?

2024-05-22 Thread Chet Ramey

On 5/22/24 12:32 AM, Zachary Santer wrote:

On Tue, May 21, 2024 at 3:06 PM Chet Ramey  wrote:


On 5/21/24 11:14 AM, Zachary Santer wrote:

On Tue, May 21, 2024 at 9:01 AM Chet Ramey  wrote:


On 5/21/24 6:17 AM, Zachary Santer wrote:


I was wondering what the relationship between the devel and master
branches was.


No mystery: the devel branch captures ongoing development, gets the latest
bug fixes, and is where new features appear. The master branch is for
stable releases.


But the devel branch won't get changes for bash versions beyond what's
in alpha right now?


I'm genuinely curious how you got that from what I said, especially since
devel has gotten changes since I released bash-5.3-alpha.


Changes I assumed would make it into master while master is bash-5.3.
It sounded like you didn't want to implement anything in devel right
now that wasn't going to make it into bash-5.3. I probably didn't
phrase that very well.


I'm not planning on including any additional new features in bash-5.3
beyond what's in alpha now. If I do some work on new features -- I don't
usually, I try to spend my time on bug fixes and release engineering --
I'll do it in a private branch that doesn't get merged until after
bash-5.3 is released.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/




Re: printf -u "$fd"?

2024-05-22 Thread Zachary Santer
On Wed, May 22, 2024 at 12:32 AM Zachary Santer  wrote:
>
> In my Rocky Linux 9.1 VM:
> $ bash --version
> GNU bash, version 5.1.8(1)-release [...]
> $ exec {fd_A}> >( cat > file_A.txt )
> $ exec {fd_B}> >( cat > file_B.txt )
> $ printf 'words\n' | tee /dev/fd/"${fd_A}" /dev/fd/"${fd_B}"
> words
> $ exec {fd_A}>&- {fd_B}>&-
> $ cat file_A.txt
> words
> $ cat file_B.txt
> words
> $ exec {fd_A}> >( tr 'w' 'W' > file_A.txt )
> $ exec {fd_B}> >( tr 'w' 'W' > file_B.txt )
> $ exec {fd_A}>&- {fd_B}>&-
> $ cat file_A.txt
> $ cat file_B.txt
> $

Yes, I missed a line there, several times actually, and then pasted that here.

$ exec {fd_A}> >( cat > file_A.txt )
$ exec {fd_B}> >( cat > file_B.txt )
$ printf 'words\n' | tee /dev/fd/"${fd_A}" /dev/fd/"${fd_B}"
words
$ exec {fd_A}>&- {fd_B}>&-
$ cat file_A.txt
words
$ cat file_B.txt
words
$ exec {fd_A}> >( tr 'w' 'W' > file_A.txt )
$ exec {fd_B}> >( tr 'w' 'W' > file_B.txt )
$ printf 'words\n' | tee /dev/fd/"${fd_A}" /dev/fd/"${fd_B}"
words
$ exec {fd_A}>&- {fd_B}>&-
$ cat file_A.txt
Words
$ cat file_B.txt
Words

I should really sit on these emails more often.



Re: printf -u "$fd"?

2024-05-21 Thread Koichi Murase
2024年5月22日(水) 0:15 Zachary Santer :
> Additionally, I was hoping the discussion of having a way to make fds
> not CLOEXEC without a loadable builtin[3][4] would get some more
> attention.

I haven't followed the discussion in the original thread for multiple
coprocs, but you can remove CLOEXEC relatively easily [1]:

  # Requires Bash >= 4.0
  function remove-cloexec {
local fd=$1
if ((fd!=0)); then
  builtin eval -- "exec 0<&$fd $fd<&- $fd<&0" &$fd $fd>&- $fd>&1" >/dev/null
fi
  }

This uses Bash's behavior of removing CLOEXEC for standard streams
0,1,2. (This behavior is probably undocumented, but I haven't checked
the document carefully). Adding CLOEXEC is more complicated and
relying on some assumptions.

[1] 
https://github.com/akinomyoga/ble.sh/blob/59787ee5528a8489e78ba09f712ed4b1f5a8e62f/src/util.sh#L3208-L3215
[2] 
https://github.com/akinomyoga/ble.sh/blob/59787ee5528a8489e78ba09f712ed4b1f5a8e62f/src/util.sh#L3114-L3207

--
Koichi



Re: printf -u "$fd"?

2024-05-21 Thread Zachary Santer
On Tue, May 21, 2024 at 3:06 PM Chet Ramey  wrote:
>
> On 5/21/24 11:14 AM, Zachary Santer wrote:
> > On Tue, May 21, 2024 at 9:01 AM Chet Ramey  wrote:
> >>
> >> On 5/21/24 6:17 AM, Zachary Santer wrote:
> >>
> >>> I was wondering what the relationship between the devel and master
> >>> branches was.
> >>
> >> No mystery: the devel branch captures ongoing development, gets the latest
> >> bug fixes, and is where new features appear. The master branch is for
> >> stable releases.
> >
> > But the devel branch won't get changes for bash versions beyond what's
> > in alpha right now?
>
> I'm genuinely curious how you got that from what I said, especially since
> devel has gotten changes since I released bash-5.3-alpha.

Changes I assumed would make it into master while master is bash-5.3.
It sounded like you didn't want to implement anything in devel right
now that wasn't going to make it into bash-5.3. I probably didn't
phrase that very well.

> > Do you create patches from devel and apply them to
> > master?
>
> Sort of, I take the code changes from the appropriate devel commit and
> generate patches against the previous patch level. Sometimes devel and
> previous-release diverge pretty  wildly, so it's not always
> straightforward. Then I apply the patches to master and push them as
> separate commits.
>
> > I guess, to be more clear, master is in the history of
> > bash-5.3-alpha,
>
> Yes, I apply the bash testing versions to the last stable release so
> people can, if they wish, see what's changed at a high level. Then I'll
> push bash-5.3-beta on top of alpha, and so on.
>
> > so I assume master will just fast-forward to that
> > point when you're happy with bash-5.3.
>
> Yes.
>
> > Meanwhile, devel is completely
> > off doing its own thing, in terms of the git commit history, Kind of
> > have a hard time wrapping my mind around that.
>
> It's how I set it up when I inherited the git repository. master has always
> been a history of release versions.

And there are some source code things that are different under devel
than in master that aren't meant to ever make it into master, if I
remember correctly? Kind of trying to come up with a better way to do
this that would allow master and devel to continue to serve the
purposes they do now. If you're at all interested.

> >>> I saw that you turned MULTIPLE_COPROCS=1 on by default
> >>> under devel, but haven't touched the somewhat more substantial changes
> >>> that sounded forthcoming, from that whole conversation.
> >>
> >> Which one is that?
> >
> > So this email[1] was just about the config-top.h change, I guess, but
> > in the prior one from you quoted there[2], you seemed to be
> > referencing only removing the coproc once both file descriptors
> > pointing to it have been closed by the user.
>
> I haven't committed to doing anything with how coprocs are reaped, and if I
> do it will certainly not be before bash-5.3.
>
>
> > Additionally, I was hoping the discussion of having a way to make fds
> > not CLOEXEC without a loadable builtin[3][4] would get some more
> > attention.
>
> I haven't returned to it, but kre's syntax is reasonable. The problem with
> doing it is described in
>
> https://lists.gnu.org/archive/html/bug-bash/2024-02/msg00194.html
>
> so it would take more work and thought, and it's not  a priority.
>
> > I want to say I tried to do 'tee /dev/fd/A /dev/fd/B' at
> > some point and didn't have the background knowledge to understand why
> > it wouldn't work.
>
> That depends on your /dev/fd implementation. There's nothing in that
> command that bash could affect.

Honestly, I might've been looking at a limitation of MSYS2.

In my Rocky Linux 9.1 VM:
$ bash --version
GNU bash, version 5.1.8(1)-release [...]
$ exec {fd_A}> >( cat > file_A.txt )
$ exec {fd_B}> >( cat > file_B.txt )
$ printf 'words\n' | tee /dev/fd/"${fd_A}" /dev/fd/"${fd_B}"
words
$ exec {fd_A}>&- {fd_B}>&-
$ cat file_A.txt
words
$ cat file_B.txt
words
$ exec {fd_A}> >( tr 'w' 'W' > file_A.txt )
$ exec {fd_B}> >( tr 'w' 'W' > file_B.txt )
$ exec {fd_A}>&- {fd_B}>&-
$ cat file_A.txt
$ cat file_B.txt
$

God knows what I was trying to do the first time, or what's going on
with the second set of procsubs there, but I didn't get "tee:
/dev/fd/N: No such file or directory" like I was expecting. Think I'll
leave this bit of the discussion to the people who know what they're
talking about.

> >>> Do you keep a list of TODOs and things under consideration somewhere?
> >>
> >> I do, it's more of the `folder of ideas' style.
> >
> > Did nosub[5] get in there? Just generalize how coproc fds get handled
> > into something that can be turned on or off for any fd.
>
> No, I don't think it's something I'm going to implement.

Eh well, I thought it would be valuable.



Re: printf -u "$fd"?

2024-05-21 Thread Chet Ramey

On 5/21/24 11:14 AM, Zachary Santer wrote:

On Tue, May 21, 2024 at 9:01 AM Chet Ramey  wrote:


On 5/21/24 6:17 AM, Zachary Santer wrote:


I was wondering what the relationship between the devel and master
branches was.


No mystery: the devel branch captures ongoing development, gets the latest
bug fixes, and is where new features appear. The master branch is for
stable releases.


But the devel branch won't get changes for bash versions beyond what's
in alpha right now? 


I'm genuinely curious how you got that from what I said, especially since
devel has gotten changes since I released bash-5.3-alpha.


Do you create patches from devel and apply them to
master?


Sort of, I take the code changes from the appropriate devel commit and
generate patches against the previous patch level. Sometimes devel and
previous-release diverge pretty  wildly, so it's not always
straightforward. Then I apply the patches to master and push them as
separate commits.


I guess, to be more clear, master is in the history of
bash-5.3-alpha,


Yes, I apply the bash testing versions to the last stable release so
people can, if they wish, see what's changed at a high level. Then I'll
push bash-5.3-beta on top of alpha, and so on.


so I assume master will just fast-forward to that
point when you're happy with bash-5.3.


Yes.


Meanwhile, devel is completely
off doing its own thing, in terms of the git commit history, Kind of
have a hard time wrapping my mind around that.


It's how I set it up when I inherited the git repository. master has always
been a history of release versions.


I saw that you turned MULTIPLE_COPROCS=1 on by default
under devel, but haven't touched the somewhat more substantial changes
that sounded forthcoming, from that whole conversation.


Which one is that?


So this email[1] was just about the config-top.h change, I guess, but
in the prior one from you quoted there[2], you seemed to be
referencing only removing the coproc once both file descriptors
pointing to it have been closed by the user.


I haven't committed to doing anything with how coprocs are reaped, and if I
do it will certainly not be before bash-5.3.



Additionally, I was hoping the discussion of having a way to make fds
not CLOEXEC without a loadable builtin[3][4] would get some more
attention.


I haven't returned to it, but kre's syntax is reasonable. The problem with
doing it is described in

https://lists.gnu.org/archive/html/bug-bash/2024-02/msg00194.html

so it would take more work and thought, and it's not  a priority.


I want to say I tried to do 'tee /dev/fd/A /dev/fd/B' at
some point and didn't have the background knowledge to understand why
it wouldn't work.


That depends on your /dev/fd implementation. There's nothing in that
command that bash could affect.


Do you keep a list of TODOs and things under consideration somewhere?


I do, it's more of the `folder of ideas' style.


Did nosub[5] get in there? Just generalize how coproc fds get handled
into something that can be turned on or off for any fd.


No, I don't think it's something I'm going to implement.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/




Re: printf -u "$fd"?

2024-05-21 Thread Zachary Santer
On Tue, May 21, 2024 at 9:01 AM Chet Ramey  wrote:
>
> On 5/21/24 6:17 AM, Zachary Santer wrote:
>
> > I was wondering what the relationship between the devel and master
> > branches was.
>
> No mystery: the devel branch captures ongoing development, gets the latest
> bug fixes, and is where new features appear. The master branch is for
> stable releases.

But the devel branch won't get changes for bash versions beyond what's
in alpha right now? Do you create patches from devel and apply them to
master? I guess, to be more clear, master is in the history of
bash-5.3-alpha, so I assume master will just fast-forward to that
point when you're happy with bash-5.3. Meanwhile, devel is completely
off doing its own thing, in terms of the git commit history, Kind of
have a hard time wrapping my mind around that.

> > I saw that you turned MULTIPLE_COPROCS=1 on by default
> > under devel, but haven't touched the somewhat more substantial changes
> > that sounded forthcoming, from that whole conversation.
>
> Which one is that?

So this email[1] was just about the config-top.h change, I guess, but
in the prior one from you quoted there[2], you seemed to be
referencing only removing the coproc once both file descriptors
pointing to it have been closed by the user.

Additionally, I was hoping the discussion of having a way to make fds
not CLOEXEC without a loadable builtin[3][4] would get some more
attention. I want to say I tried to do 'tee /dev/fd/A /dev/fd/B' at
some point and didn't have the background knowledge to understand why
it wouldn't work.

> > So I take it
> > MULTIPLE_COPROCS=1 will be enabled in bash-5.3 but other potential
> > changes would come later?
>
> Probably, yes.
>
> > Do you keep a list of TODOs and things under consideration somewhere?
>
> I do, it's more of the `folder of ideas' style.

Did nosub[5] get in there? Just generalize how coproc fds get handled
into something that can be turned on or off for any fd.

[1] https://lists.gnu.org/archive/html/bug-bash/2024-04/msg00149.html
[2] https://lists.gnu.org/archive/html/bug-bash/2024-04/msg00105.html
[3] https://lists.gnu.org/archive/html/bug-bash/2024-04/msg00085.html
[4] https://lists.gnu.org/archive/html/bug-bash/2024-04/msg00100.html
[5] https://lists.gnu.org/archive/html/bug-bash/2024-04/msg00087.html



Re: printf -u "$fd"?

2024-05-21 Thread Chet Ramey

On 5/21/24 6:17 AM, Zachary Santer wrote:


I was wondering what the relationship between the devel and master
branches was. 


No mystery: the devel branch captures ongoing development, gets the latest
bug fixes, and is where new features appear. The master branch is for
stable releases.


I saw that you turned MULTIPLE_COPROCS=1 on by default
under devel, but haven't touched the somewhat more substantial changes
that sounded forthcoming, from that whole conversation. 


Which one is that?


So I take it
MULTIPLE_COPROCS=1 will be enabled in bash-5.3 but other potential
changes would come later?


Probably, yes.


Do you keep a list of TODOs and things under consideration somewhere?


I do, it's more of the `folder of ideas' style.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/




Re: printf -u "$fd"?

2024-05-21 Thread Zachary Santer
On Mon, May 20, 2024 at 3:03 PM Chet Ramey  wrote:
>
> On 5/17/24 10:53 PM, Zachary Santer wrote:
>
> > So here's another tangent, but has it been considered to add an option
> > to the printf builtin to print to a given file descriptor, rather than
> > stdout? If printing to a number of different file descriptors in
> > succession, such an option would appear to have all the same benefits
> > as read's -u option.
>
> It doesn't actually save anything; it's just syntactic sugar. Since
> `printf' uses stdio internally (as opposed to `read', which uses the
> supplied file descriptor directly), you're still going to have to dup2
> it internally somewhere. dprintf could do some of the work here, and bash
> has a replacement for systems where it's missing, but that's more than I
> want to change before bash-5.3 comes out. Maybe after that.

I was wondering what the relationship between the devel and master
branches was. I saw that you turned MULTIPLE_COPROCS=1 on by default
under devel, but haven't touched the somewhat more substantial changes
that sounded forthcoming, from that whole conversation. So I take it
MULTIPLE_COPROCS=1 will be enabled in bash-5.3 but other potential
changes would come later?

Do you keep a list of TODOs and things under consideration somewhere?



Re: printf -u "$fd"?

2024-05-20 Thread Chet Ramey

On 5/17/24 10:53 PM, Zachary Santer wrote:


So here's another tangent, but has it been considered to add an option
to the printf builtin to print to a given file descriptor, rather than
stdout? If printing to a number of different file descriptors in
succession, such an option would appear to have all the same benefits
as read's -u option.


It doesn't actually save anything; it's just syntactic sugar. Since
`printf' uses stdio internally (as opposed to `read', which uses the
supplied file descriptor directly), you're still going to have to dup2
it internally somewhere. dprintf could do some of the work here, and bash
has a replacement for systems where it's missing, but that's more than I
want to change before bash-5.3 comes out. Maybe after that.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: printf -u "$fd"?

2024-05-19 Thread alex xmb sw ratchev
On Sun, May 19, 2024, 20:11 Kerin Millar  wrote:

> On Sun, 19 May 2024, at 5:08 PM, alex xmb sw ratchev wrote:
> > On Sat, May 18, 2024, 04:54 Zachary Santer  wrote:
> >
> >> Was «difference between read -u fd and read <&"$fd"» on
> help-b...@gnu.org
> >>
> >> On Thu, May 16, 2024 at 12:51 AM Kerin Millar 
> wrote:
> >> >
> >> > On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote:
> >> > > Hi,
> >> > >
> >> > > It appears to me that read -u fd and read <&"$fd" achieve the same
> >> > > result. But I may miss corner cases when they may be different.
> >> > >
> >> > > Is it true that they are exactly the same?
> >> >
> >> > They are not exactly the same. To write read -u fd is to instruct the
> >> read builtin to read directly from the specified file descriptor. To
> write
> >> read <&"$fd" entails one invocation of the dup2 syscall to duplicate the
> >> specified file descriptor to file descriptor #0 and another invocation
> to
> >> restore it once read has concluded. That's measurably slower where
> looping
> >> over read.
> >>
> >> So here's another tangent, but has it been considered to add an option
> >> to the printf builtin to print to a given file descriptor, rather than
> >> stdout? If printing to a number of different file descriptors in
> >> succession, such an option would appear to have all the same benefits
> >> as read's -u option.
> >>
> >
> > id like a printf -u
> >
> > Zack
>
> I understand that you have pledged to use broken email software for the
> rest of time but could you at least refrain from posting emails that look
> as though they were signed off by someone other than yourself? It's not
> cricket.
>

what , sorry ?
is it the case everyone tells me i code obfuscated while at all i never did
? just cause i shortened varnames ?
u speak about ' zack ' at end of mail display ? ye buggy email sw .. i del
< and do 3 newlines and in the middle i enter text ..

-- 
> Kerin Millar
>


Re: printf -u "$fd"?

2024-05-19 Thread Kerin Millar
On Sun, 19 May 2024, at 5:08 PM, alex xmb sw ratchev wrote:
> On Sat, May 18, 2024, 04:54 Zachary Santer  wrote:
>
>> Was «difference between read -u fd and read <&"$fd"» on help-b...@gnu.org
>>
>> On Thu, May 16, 2024 at 12:51 AM Kerin Millar  wrote:
>> >
>> > On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote:
>> > > Hi,
>> > >
>> > > It appears to me that read -u fd and read <&"$fd" achieve the same
>> > > result. But I may miss corner cases when they may be different.
>> > >
>> > > Is it true that they are exactly the same?
>> >
>> > They are not exactly the same. To write read -u fd is to instruct the
>> read builtin to read directly from the specified file descriptor. To write
>> read <&"$fd" entails one invocation of the dup2 syscall to duplicate the
>> specified file descriptor to file descriptor #0 and another invocation to
>> restore it once read has concluded. That's measurably slower where looping
>> over read.
>>
>> So here's another tangent, but has it been considered to add an option
>> to the printf builtin to print to a given file descriptor, rather than
>> stdout? If printing to a number of different file descriptors in
>> succession, such an option would appear to have all the same benefits
>> as read's -u option.
>>
>
> id like a printf -u
>
> Zack

I understand that you have pledged to use broken email software for the rest of 
time but could you at least refrain from posting emails that look as though 
they were signed off by someone other than yourself? It's not cricket.

-- 
Kerin Millar



Re: printf -u "$fd"?

2024-05-19 Thread alex xmb sw ratchev
On Sat, May 18, 2024, 04:54 Zachary Santer  wrote:

> Was «difference between read -u fd and read <&"$fd"» on help-b...@gnu.org
>
> On Thu, May 16, 2024 at 12:51 AM Kerin Millar  wrote:
> >
> > On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote:
> > > Hi,
> > >
> > > It appears to me that read -u fd and read <&"$fd" achieve the same
> > > result. But I may miss corner cases when they may be different.
> > >
> > > Is it true that they are exactly the same?
> >
> > They are not exactly the same. To write read -u fd is to instruct the
> read builtin to read directly from the specified file descriptor. To write
> read <&"$fd" entails one invocation of the dup2 syscall to duplicate the
> specified file descriptor to file descriptor #0 and another invocation to
> restore it once read has concluded. That's measurably slower where looping
> over read.
>
> So here's another tangent, but has it been considered to add an option
> to the printf builtin to print to a given file descriptor, rather than
> stdout? If printing to a number of different file descriptors in
> succession, such an option would appear to have all the same benefits
> as read's -u option.
>

id like a printf -u

Zack
>
>


Re: printf -u "$fd"?

2024-05-19 Thread Kerin Millar
On Sat, 18 May 2024, at 3:53 AM, Zachary Santer wrote:
> Was «difference between read -u fd and read <&"$fd"» on help-b...@gnu.org
>
> On Thu, May 16, 2024 at 12:51 AM Kerin Millar  wrote:
>>
>> On Thu, 16 May 2024, at 3:25 AM, Peng Yu wrote:
>> > Hi,
>> >
>> > It appears to me that read -u fd and read <&"$fd" achieve the same
>> > result. But I may miss corner cases when they may be different.
>> >
>> > Is it true that they are exactly the same?
>>
>> They are not exactly the same. To write read -u fd is to instruct the read 
>> builtin to read directly from the specified file descriptor. To write read 
>> <&"$fd" entails one invocation of the dup2 syscall to duplicate the 
>> specified file descriptor to file descriptor #0 and another invocation to 
>> restore it once read has concluded. That's measurably slower where looping 
>> over read.
>
> So here's another tangent, but has it been considered to add an option
> to the printf builtin to print to a given file descriptor, rather than
> stdout? If printing to a number of different file descriptors in
> succession, such an option would appear to have all the same benefits
> as read's -u option.

Until now, not that I am aware of. I normally apply a redirection to a group 
command in such cases, though it still results in exactly two dup2 calls.

-- 
Kerin Millar