Re: multi-threaded compiling

> > Date: Wed, 13 Mar 2024 22:52:16 >> > From: Mischa Baars >> > To: alex xmb sw ratchev >> > Cc: psm...@gnu.org, bug-bash , help-m...@gnu.org >> > Subject: Re: multi-threaded compiling >> > >> > I found another mistake of mine: >>

Re: multi-threaded compiling

: > On Wed, 13 Mar 2024, Mischa Baars wrote: > > > Date: Wed, 13 Mar 2024 22:52:16 > > From: Mischa Baars > > To: alex xmb sw ratchev > > Cc: psm...@gnu.org, bug-bash , help-m...@gnu.org > > Subject: Re: multi-threaded compiling > > > > I found another

Re: multi-threaded compiling

I found another mistake of mine: #ifndef __STRINGIZED__ should be #if __STRINGIZED__ == 0 or it will always go for the second statement in the conditional. Can someone please correct my script??? Things are starting to itch. On Wed, Mar 13, 2024 at 8:44 AM alex xmb sw ratchev wrote: > > >

Re: multi-threaded compiling

On Wed, Mar 13, 2024, 08:26 Mischa Baars wrote: > On Tue, Mar 12, 2024 at 10:00 PM Paul Smith wrote: > > > On Tue, 2024-03-12 at 13:37 +0100, Mischa Baars wrote: > > > > I'd still like to hear why you aren't simply using "make -j". > > > > > > That's because I don't want to define static

Re: multi-threaded compiling

On Wed, Mar 13, 2024 at 4:14 AM Martin D Kealey wrote: > > On Mon, Mar 11, 2024 at 8:20 PM Chet Ramey wrote: >> > On 3/11/24 2:50 PM, Mischa Baars wrote: >> > > Which sort of brings us back to the original question I suppose. Who >> does >> > > that line of code function from a script and why

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 10:00 PM Paul Smith wrote: > On Tue, 2024-03-12 at 13:37 +0100, Mischa Baars wrote: > > > I'd still like to hear why you aren't simply using "make -j". > > > > That's because I don't want to define static compile and link targets > > for every new project I start. The

Re: multi-threaded compiling

> On Mon, Mar 11, 2024 at 8:20 PM Chet Ramey wrote: > > On 3/11/24 2:50 PM, Mischa Baars wrote: > > > Which sort of brings us back to the original question I suppose. Who > does > > > that line of code function from a script and why does it fail from the > > > command line? > > > > Job control

Re: multi-threaded compiling

On Tue, 2024-03-12 at 13:37 +0100, Mischa Baars wrote: > > I'd still like to hear why you aren't simply using "make -j". > > That's because I don't want to define static compile and link targets > for every new project I start. The Makefile in question contains only > a few lines of code and some

Re: multi-threaded compiling

On 3/12/24 4:23 AM, Mischa Baars wrote: So... how do I fix it? If you want double quotes in a variable expansion to have some meaning to the parser, you need to run the expanded value through the parser, using something like `eval'. Or you figure out a different way to glue the arguments

Re: multi-threaded compiling

On 3/12/24 7:29 AM, Greg Wooledge wrote: hobbit:~$ bash -c 'set +m; for i in {0..3}; do sleep 1 & done; for i in {0..3}; do wait -n -p pid; e=$?; printf "pid %s status %s\n" "$pid" "$e"; done' pid 530407 status 0 pid 530410 status 0 pid status 127 pid status 127 Thanks for this example.

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 12:49 PM Greg Wooledge wrote: > On Tue, Mar 12, 2024 at 09:42:22AM +0100, Mischa Baars wrote: > > Here's the script and the Makefile using "printf '<%s>'": > > Sadly, your mail user agent chose to attach "Makefile" with content-type > application/octet-stream, which my

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 12:30 PM Greg Wooledge wrote: > On Tue, Mar 12, 2024 at 02:15:38PM +0700, Robert Elz wrote: > > Date:Mon, 11 Mar 2024 17:25:57 -0400 > > From:Chet Ramey > > Message-ID: <322e10a6-3808-49be-aa9d-a1d367a90...@case.edu> > > > > | OK, here's

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 12:19 PM Greg Wooledge wrote: > On Tue, Mar 12, 2024 at 10:33:36AM +0100, Mischa Baars wrote: > > bash -c 'set +m; seconds=1; for (( i=0;i<32;i++ )); do exit ${i} & done; > > sleep ${seconds}; for (( i=0;i<32;i++ )); do wait -p pid; e=${?}; echo > > "$(printf %3u ${i})

Re: multi-threaded compiling

On Tue, Mar 12, 2024, 12:49 Greg Wooledge wrote: > On Tue, Mar 12, 2024 at 09:42:22AM +0100, Mischa Baars wrote: > > Here's the script and the Makefile using "printf '<%s>'": > > Sadly, your mail user agent chose to attach "Makefile" with content-type > application/octet-stream, which my MUA

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 09:42:22AM +0100, Mischa Baars wrote: > Here's the script and the Makefile using "printf '<%s>'": Sadly, your mail user agent chose to attach "Makefile" with content-type application/octet-stream, which my MUA (mutt) refuses to show inline, or to include in a reply as

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 02:15:38PM +0700, Robert Elz wrote: > Date:Mon, 11 Mar 2024 17:25:57 -0400 > From:Chet Ramey > Message-ID: <322e10a6-3808-49be-aa9d-a1d367a90...@case.edu> > > | OK, here's the longer answer. When the shell is interactive, and job > |

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 10:33:36AM +0100, Mischa Baars wrote: > bash -c 'set +m; seconds=1; for (( i=0;i<32;i++ )); do exit ${i} & done; > sleep ${seconds}; for (( i=0;i<32;i++ )); do wait -p pid; e=${?}; echo > "$(printf %3u ${i}) pid ${pid} exit ${e}"; done;' "wait -p pid" is not correct here.

Re: multi-threaded compiling

Date:Tue, 12 Mar 2024 18:32:19 +1000 From:Martin D Kealey Message-ID: | You don't need '-n' when you specify a PID; the fix is simply to remove it. That's OK, when there is exactly one PID, but for a task like this, wait -n -p pid PID1 PID2 PID3 ...

Re: multi-threaded compiling

| I think you forgot to look at the actual code in the Makefile here. No, I didn't forget, I never intended to look, unpacking attached tar.gz files is more effort than I'm willing to undertake. | Every '$' is written as a '$$', exactly as you say it should. OK, I guessed incorrectly ...

Re: multi-threaded compiling

Got it!!! bash -c 'set +m; seconds=5; for (( i=0;i<32;i++ )); do exit ${i} & pid[${i}]=${!}; done; sleep ${seconds}; for (( i=0;i<32;i++ )); do wait ${pid[${i}]}; e=${?}; echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;' One problem solved :) The problem was exactly the -n, and had

Re: multi-threaded compiling

Oh, sorry. Without the -n that is. bash -c 'set +m; seconds=1; for (( i=0;i<32;i++ )); do exit ${i} & done; sleep ${seconds}; for (( i=0;i<32;i++ )); do wait -p pid; e=${?}; echo "$(printf %3u ${i}) pid ${pid} exit ${e}"; done;' bash -c 'set -m; seconds=1; for (( i=0;i<32;i++ )); do exit ${i} &

Re: multi-threaded compiling

Hi Chet, The only thing I can find about this behavior is the set +m / set -m option, which disables / enables the notifications, but has no effect otherwise. The line from the script won't execute from the command line with either option, neither enclosed with a 'bash -c' as in: set +m; bash

Re: multi-threaded compiling

Here's the script and the Makefile using "printf '<%s>'": On Tue, Mar 12, 2024 at 9:32 AM Martin D Kealey wrote: > In section one, the problem is that "wait -n" does not do what you think > it does. (Lots of us think this behaviour is broken, and it may be fixed in > an upcoming version of

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 9:32 AM Martin D Kealey wrote: > In section one, the problem is that "wait -n" does not do what you think > it does. (Lots of us think this behaviour is broken, and it may be fixed in > an upcoming version of Bash.) You don't need '-n' when you specify a PID; > the fix is

Re: multi-threaded compiling

In section one, the problem is that "wait -n" does not do what you think it does. (Lots of us think this behaviour is broken, and it may be fixed in an upcoming version of Bash.) You don't need '-n' when you specify a PID; the fix is simply to remove it. In section two, the problem is that quote

Re: multi-threaded compiling

On Tue, Mar 12, 2024, 09:26 Mischa Baars wrote: > On Mon, Mar 11, 2024 at 10:26 PM Chet Ramey wrote: > > > On 3/11/24 3:44 PM, Mischa Baars wrote: > > > On Mon, 11 Mar 2024, 20:20 Chet Ramey, > > > wrote: > > > > > > On 3/11/24 2:50 PM, Mischa Baars wrote: > > >

Re: multi-threaded compiling

On Mon, Mar 11, 2024 at 10:26 PM Chet Ramey wrote: > On 3/11/24 3:44 PM, Mischa Baars wrote: > > On Mon, 11 Mar 2024, 20:20 Chet Ramey, > > wrote: > > > > On 3/11/24 2:50 PM, Mischa Baars wrote: > > > Which sort of brings us back to the original question I

Re: multi-threaded compiling

On Tue, Mar 12, 2024 at 8:16 AM Robert Elz wrote: > This whole discussuon has been a total mess. > > First the original message didn't give nearly enough info. > That might have been in the attachment, but no-one should be > expecting people to do any needless work when you're asking > for

Re: multi-threaded compiling

This whole discussuon has been a total mess. First the original message didn't give nearly enough info. That might have been in the attachment, but no-one should be expecting people to do any needless work when you're asking for (free) assistance - if you want help you need to make it as easy as

Re: multi-threaded compiling

a version with success or fail handling ~ $ bash xmb.smallt.2 2/2 1/1 ~ $ cat xmb.smallt.2 #!/bin/bash unset -v success fail cmd a e run() { # uaage , $1 successcase cmd , $2 fail cmd , $3 and further actual cmd to run local IFS=' ' eval "${*:3} &" success[$!]=$1 fail[$!]=$2 cmd[$!]=${*:3} }

Re: multi-threaded compiling

On Mon, Mar 11, 2024, 22:40 alex xmb sw ratchev wrote: > > > On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev wrote: > >> ~ $ bash xmb.smallt >> pid 14333 cmd t2 returned 3 >> pid 14332 cmd sleep 1 returned 0 >> >> ~ $ cat xmb.smallt >> #!/bin/bash >> >> run() { >> local IFS=' ' run=$* >> eval

Re: multi-threaded compiling

On Mon, Mar 11, 2024, 22:40 alex xmb sw ratchev wrote: > > > On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev wrote: > >> ~ $ bash xmb.smallt >> pid 14333 cmd t2 returned 3 >> pid 14332 cmd sleep 1 returned 0 >> >> ~ $ cat xmb.smallt >> #!/bin/bash >> >> run() { >> local IFS=' ' run=$* >> eval

Re: multi-threaded compiling

On Mon, Mar 11, 2024, 22:36 alex xmb sw ratchev wrote: > ~ $ bash xmb.smallt > pid 14333 cmd t2 returned 3 > pid 14332 cmd sleep 1 returned 0 > > ~ $ cat xmb.smallt > #!/bin/bash > > run() { > local IFS=' ' run=$* > eval "$run &" > me[$!]=$run > } > alternativly to mention aliases usage

Re: multi-threaded compiling

~ $ bash xmb.smallt pid 14333 cmd t2 returned 3 pid 14332 cmd sleep 1 returned 0 ~ $ cat xmb.smallt #!/bin/bash run() { local IFS=' ' run=$* eval "$run &" me[$!]=$run } wa() { local pid wait -n -p pid printf %s\\n "pid $pid cmd ${me[pid]} returned $?" } t2() { sleep .75 return 3 } run

Re: multi-threaded compiling

On 3/11/24 3:44 PM, Mischa Baars wrote: On Mon, 11 Mar 2024, 20:20 Chet Ramey, > wrote: On 3/11/24 2:50 PM, Mischa Baars wrote: > Which sort of brings us back to the original question I suppose. Who does > that line of code function from a script

Re: multi-threaded compiling

On Mon, Mar 11, 2024 at 10:19:26PM +0100, Mischa Baars wrote: > On Mon, 11 Mar 2024, 21:08 Kerin Millar, wrote: > > The pid can be obtained with the -p option, as of 5.1. Below is a > > synthetic example of how it might be put into practice. I'd forgotten about that one. A recent addition, and

Re: multi-threaded compiling

On Mon, 11 Mar 2024, 21:08 Kerin Millar, wrote: > On Mon, 11 Mar 2024 15:36:48 -0400 > Greg Wooledge wrote: > > > > On Mon, Mar 11, 2024, 20:13 Mischa Baars > > > > wrote: > > > > > > > Also I don't think that gives you an exit status for each 'exit $i' > > > > started. I need that exit

Re: multi-threaded compiling

Like this: seconds=1; for (( i=0;i<32;i++ )); do exit ${i} & done; sleep ${seconds}; for (( i=0;i<32;i++ )); do wait -np pid; e=${?}; echo "$(printf %3u ${i}) pid ${pid} exit ${e}"; done; which again only works when called from a script. I have been looking for the -p option. Guess I overlooked

Re: multi-threaded compiling

On Mon, 11 Mar 2024 15:36:48 -0400 Greg Wooledge wrote: > > On Mon, Mar 11, 2024, 20:13 Mischa Baars > > wrote: > > > > > Also I don't think that gives you an exit status for each 'exit $i' > > > started. I need that exit status. > > "wait -n" without a PID won't help you, then. You don't

Re: multi-threaded compiling

bash s wait returns the exit code of the proc from the job On Mon, Mar 11, 2024, 20:52 Mischa Baars wrote: > On Mon, 11 Mar 2024, 20:36 Greg Wooledge, wrote: > > > > On Mon, Mar 11, 2024, 20:13 Mischa Baars > > > > wrote: > > > > > > > Also I don't think that gives you an exit status for each

Re: multi-threaded compiling

On Mon, 11 Mar 2024, 20:36 Greg Wooledge, wrote: > > On Mon, Mar 11, 2024, 20:13 Mischa Baars > > wrote: > > > > > Also I don't think that gives you an exit status for each 'exit $i' > > > started. I need that exit status. > > "wait -n" without a PID won't help you, then. You don't get the PID

Re: multi-threaded compiling

On 3/11/24 2:46 PM, Mischa Baars wrote: You mean: for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do wait -n; echo $?; done; Because this doesn't and to be honest, I needed the pid and its index to retrieve gcc's output from a log file array afterwards. If you want

Re: multi-threaded compiling

On Mon, 11 Mar 2024, 20:20 Chet Ramey, wrote: > On 3/11/24 2:50 PM, Mischa Baars wrote: > > Which sort of brings us back to the original question I suppose. Who does > > that line of code function from a script and why does it fail from the > > command line? > > Job control and when the shell

Re: multi-threaded compiling

On Mon, 2024-03-11 at 15:36 -0400, Greg Wooledge wrote: > You won't be able to take advantage of "wait -n"'s ability to react > to the first job that finishes.  You'll end up reaping each job in > the order they started, not the order they finished. It feels to me like you're trying to reproduce

Re: multi-threaded compiling

> On Mon, Mar 11, 2024, 20:13 Mischa Baars > wrote: > > > Also I don't think that gives you an exit status for each 'exit $i' > > started. I need that exit status. "wait -n" without a PID won't help you, then. You don't get the PID or job ID that terminated, and you don't get the exit status.

Re: multi-threaded compiling

afik wait will exit with that $? of returned On Mon, Mar 11, 2024, 20:13 Mischa Baars wrote: > That's not really an answer to the question. > > Also I don't think that gives you an exit status for each 'exit $i' > started. I need that exit status. > > > On Mon, 11 Mar 2024, 20:03 alex xmb sw

Re: multi-threaded compiling

On 3/11/24 2:50 PM, Mischa Baars wrote: Which sort of brings us back to the original question I suppose. Who does that line of code function from a script and why does it fail from the command line? Job control and when the shell notifies the user about job completion, most likely, two of the

Re: multi-threaded compiling

That's not really an answer to the question. Also I don't think that gives you an exit status for each 'exit $i' started. I need that exit status. On Mon, 11 Mar 2024, 20:03 alex xmb sw ratchev, wrote: > > > On Mon, Mar 11, 2024, 20:03 alex xmb sw ratchev wrote: > >> the logic between my

Re: multi-threaded compiling

On Mon, Mar 11, 2024, 20:03 alex xmb sw ratchev wrote: > the logic between my code > > 1 threads_max > 2 loop > 3 inside loop , do if run is > than threads_max then wait -n one > then 4 spawn thread > 3 if run isnt more than max , simply ignore and spawn thread in next cmd i dont get ur points

Re: multi-threaded compiling

the logic between my code 1 threads_max 2 loop 3 inside loop , do if run is > than threads_max then wait -n one then 4 spawn thread i dont get ur points On Mon, Mar 11, 2024, 19:55 Mischa Baars wrote: > Sorry. I mean: > > for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ ));

Re: multi-threaded compiling

Sorry. I mean: for (( i=0; i<32; i++ )); do exit $i & done; for (( i=0; i<32; i++ )); do wait -n; echo $?; done; doesn't function. With an ampersand instead of a semicolon. Why does it function when called from a script and why does it fail when called from the command line? On Mon, Mar 11,

Re: multi-threaded compiling

Which sort of brings us back to the original question I suppose. Who does that line of code function from a script and why does it fail from the command line? My guess was that the same thing makes this line fail from the Makefile. On Mon, Mar 11, 2024 at 7:46 PM Mischa Baars wrote: > You mean:

Re: multi-threaded compiling

You mean: for (( i=0; i<32; i++ )); do exit $i & wait -n; echo $?; done; with one command and one wait in a single loop. And this does execute on the command line. How interesting! for (( i=0; i<32; i++ )); do exit $i; done; for (( i=0; i<32; i++ )); do wait -n; echo $?; done; Because this

Re: multi-threaded compiling

On Mon, 2024-03-11 at 19:37 +0100, alex xmb sw ratchev wrote: > /data/data/com.termux/files/usr/bin/sh: 1: Syntax error: Bad for loop > variable This is because of the issue I mentioned in my initial reply. This invocation is using /bin/sh which is a POSIX shell but is not bash; probably it's

Re: multi-threaded compiling

other output ~/2024031100 - gnu questions/one $ make SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!}; done; sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]}; e=${?}; echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done;

Re: multi-threaded compiling

On Mon, Mar 11, 2024, 19:25 alex xmb sw ratchev wrote: > > > On Mon, Mar 11, 2024, 19:22 Mischa Baars > wrote: > >> On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev >> wrote: >> >>> i also completly dont get ur issue >>> >>> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r= >>> >>> while [[ -v

Re: multi-threaded compiling

On Mon, Mar 11, 2024 at 07:22:39PM +0100, Mischa Baars wrote: > On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev > wrote: > > > i also completly dont get ur issue > > > > f=( a.c b.c .. ) threads=$( nproc ) i=-1 r= > > > > while [[ -v f[++i] ]] ; do > > (( ++r > threads )) && > > wait -n >

Re: multi-threaded compiling

On Mon, Mar 11, 2024 at 7:14 PM Greg Wooledge wrote: > On Mon, Mar 11, 2024 at 06:51:54PM +0100, Mischa Baars wrote: > > SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!}; > done; sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]}; > e=${?}; echo "$(printf %3u

Re: multi-threaded compiling

Hi Greg, Good point. One for you :) Cheerz, Mischa. On Mon, Mar 11, 2024 at 7:14 PM Greg Wooledge wrote: > On Mon, Mar 11, 2024 at 06:51:54PM +0100, Mischa Baars wrote: > > SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!}; > done; sleep ${SECONDS}; for (( i=0;i<32;i++ ));

Re: multi-threaded compiling

On Mon, Mar 11, 2024, 19:22 Mischa Baars wrote: > On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev > wrote: > >> i also completly dont get ur issue >> >> f=( a.c b.c .. ) threads=$( nproc ) i=-1 r= >> >> while [[ -v f[++i] ]] ; do >> (( ++r > threads )) && >> wait -n >> gcc -c "${f[i]}" &

Re: multi-threaded compiling

On Mon, Mar 11, 2024 at 6:22 PM alex xmb sw ratchev wrote: > i also completly dont get ur issue > > f=( a.c b.c .. ) threads=$( nproc ) i=-1 r= > > while [[ -v f[++i] ]] ; do > (( ++r > threads )) && > wait -n > gcc -c "${f[i]}" & > done > How nice! wait -n exit 1 & echo $? You got me

Re: multi-threaded compiling

On Mon, Mar 11, 2024 at 06:51:54PM +0100, Mischa Baars wrote: > SECONDS=5; for (( i=0;i<32;i++ )); do { exit ${i}; } & pid[${i}]=${!}; done; > sleep ${SECONDS}; for (( i=0;i<32;i++ )); do wait -n ${pid[${i}]}; e=${?}; > echo "$(printf %3u ${i}) pid ${pid[${i}]} exit ${e}"; done; > /bin/bash:

Re: multi-threaded compiling

Oh, ok. I have executed the examples for you now. What I expected was also in the first mail: 1) The one.make.sh.log has the expected output. The same commands executed from a Makefile (one.make.log) do not behave as expected. This also holds for this command executed from the command line. 2)

Re: multi-threaded compiling

On Mon, 2024-03-11 at 18:14 +0100, Mischa Baars wrote: > The actual command invocations (a Makefile, a make.sh script) can be > found in the attachment, as indicated on the first line of the mail. > In the attachment there are two directories, one and two, belonging > to 1) and 2) respectively.

Re: multi-threaded compiling

i also completly dont get ur issue f=( a.c b.c .. ) threads=$( nproc ) i=-1 r= while [[ -v f[++i] ]] ; do (( ++r > threads )) && wait -n gcc -c "${f[i]}" & done On Mon, Mar 11, 2024, 18:16 Mischa Baars wrote: > Hello Paul, > > It seems I'm awake a little longer than you are. > > The

Re: multi-threaded compiling

Hello Paul, It seems I'm awake a little longer than you are. The second paragraph as you see it, belongs to 1) The fourth paragraph as you see it, belongs to 2) The actual command invocations (a Makefile, a make.sh script) can be found in the attachment, as indicated on the first line of the

Re: multi-threaded compiling

On Mon, 2024-03-11 at 09:56 +0100, Mischa Baars wrote: > I've attached the problem. It consists of two parts: > > 1) multi-threaded bash script and / or multi-threaded Makefile > > Running bash script functions as expected, but executing the same > line of code with make and / or the command

multi-threaded compiling

Hi, I'd like to compile a project of mine using multiple logical cores. I've attached the problem. It consists of two parts: 1) multi-threaded bash script and / or multi-threaded Makefile Running bash script functions as expected, but executing the same line of code with make and / or the