Re: bmake inefficiencies

2021-02-05 Thread Simon J. Gerraty
Robert Elz wrote: > [External Email. Be cautious of content] > > > Date:Fri, 5 Feb 2021 10:19:10 -0800 > From:"Simon J. Gerraty" > Message-ID: <38484.1612549...@kaos.jnpr.net> > > | The open in mkstemp can fail due to EINTR. > | That's the only reason I see

Re: bmake inefficiencies

2021-02-05 Thread Simon J. Gerraty
Mateusz Guzik wrote: > More importantly though, folding all of this into one routine will > facilitate using O_TMPFILE. It's a Linux-specific (for now) flag to > create a file without having to supply a random pattern (and having to > unlink it later, like you do here). For bmake, O_TMPFILE

Re: bmake inefficiencies

2021-02-05 Thread Robert Elz
Date:Fri, 5 Feb 2021 10:19:10 -0800 From:"Simon J. Gerraty" Message-ID: <38484.1612549...@kaos.jnpr.net> | The open in mkstemp can fail due to EINTR. | That's the only reason I see for blocking signals. Surely the reason would have been to avoid leaving stray

Re: bmake inefficiencies

2021-02-05 Thread Mateusz Guzik
On 2/5/21, Simon J. Gerraty wrote: > Mateusz Guzik wrote: >> >> is the signal stuff really necessary? >> > >> > It avoids the need to loop dealing with interupts. >> > The name seems missleading should be block not lock. >> > >> >> I know what it does, it popped up on truss. >> >> My point is

Re: bmake inefficiencies

2021-02-05 Thread Simon J. Gerraty
Mateusz Guzik wrote: > >> is the signal stuff really necessary? > > > > It avoids the need to loop dealing with interupts. > > The name seems missleading should be block not lock. > > > > I know what it does, it popped up on truss. > > My point is that there are no callers of mkTempFile from a

Re: bmake inefficiencies

2021-02-05 Thread Mateusz Guzik
Responding to the original mail due to several follow ups. Thanks for fixups so far. Another note related to JobOpenTmpFile: JobSigLock(); tfd = mkTempFile(TMPPAT, ); if (!DEBUG(SCRIPT)) (void)eunlink(tfile); JobSigUnlock(); is the signal stuff

Re: bmake inefficiencies

2021-02-01 Thread Simon J. Gerraty
Jason Thorpe wrote: > > I chose int (which is what sig_atomic_t is) for portability. > > sig_atomic_t has been around for a fairly long time now, and NOT using > it has a negative portability impact. Yes I'll change it - I can deal with making sig_atomic_t available if missing - that's the

Re: bmake inefficiencies

2021-02-01 Thread Jason Thorpe
> On Jan 31, 2021, at 12:41 PM, Simon J. Gerraty wrote: > > David Holland wrote: >> "static volatile sig_atomic_t reap_children;" > > I chose int (which is what sig_atomic_t is) for portability. sig_atomic_t has been around for a fairly long time now, and NOT using it has a negative

Re: bmake inefficiencies

2021-02-01 Thread Robert Elz
Date:Mon, 1 Feb 2021 09:04:54 +0100 (GMT+01:00) From:Roland Illig Message-ID: | According to https://en.cppreference.com/w/c/program/sig_atomic_t, | it has been available since C90, so don't worry about portability. Further, even if you're dealing with an

Re: bmake inefficiencies

2021-02-01 Thread Roland Illig
01.02.2021 07:58:26 Simon J. Gerraty : > Martin Husemann wrote: > >> [External Email. Be cautious of content] >> >> >> On Sun, Jan 31, 2021 at 12:41:47PM -0800, Simon J. Gerraty wrote: >>> David Holland wrote: "static volatile sig_atomic_t reap_children;" >>> >>> I chose int (which is

Re: bmake inefficiencies

2021-01-31 Thread Martin Husemann
On Sun, Jan 31, 2021 at 10:52:03PM -0800, Simon J. Gerraty wrote: > > No, it is not int - several NetBSD architectures use long. > > Sorry, the one I checked as int, but I don't see that it should really > matter? int should be the type that any architecture deals with most > naturally. You

Re: bmake inefficiencies

2021-01-31 Thread Simon J. Gerraty
Martin Husemann wrote: > [External Email. Be cautious of content] > > > On Sun, Jan 31, 2021 at 12:41:47PM -0800, Simon J. Gerraty wrote: > > David Holland wrote: > > > "static volatile sig_atomic_t reap_children;" > > > > I chose int (which is what sig_atomic_t is) for portability. > > No,

Re: bmake inefficiencies

2021-01-31 Thread Martin Husemann
On Sun, Jan 31, 2021 at 12:41:47PM -0800, Simon J. Gerraty wrote: > David Holland wrote: > > "static volatile sig_atomic_t reap_children;" > > I chose int (which is what sig_atomic_t is) for portability. No, it is not int - several NetBSD architectures use long. Martin

Re: bmake inefficiencies

2021-01-31 Thread Simon J. Gerraty
David Holland wrote: > It might be necessary to try waiting a second time, because it might > not be able to know how many children have exited (depends if they all Yes, SIGCHLD tells you that at least one children died. So looping on waitpid is needed. > trigger some EOF or just most of them)

Re: bmake inefficiencies

2021-01-31 Thread Simon J. Gerraty
Simon J. Gerraty wrote: > David Holland wrote: > > "static volatile sig_atomic_t reap_children;" > > I chose int (which is what sig_atomic_t is) for portability. > > > but also, I would expect that this childExitJob.outPipe can go away. > > Possibly, the claimed purpose is to cause

Re: bmake inefficiencies

2021-01-31 Thread Simon J. Gerraty
David Holland wrote: > "static volatile sig_atomic_t reap_children;" I chose int (which is what sig_atomic_t is) for portability. > but also, I would expect that this childExitJob.outPipe can go away. Possibly, the claimed purpose is to cause poll/select to return.

Re: bmake inefficiencies

2021-01-31 Thread Roland Illig
On 31.01.2021 19:28, David Holland wrote: (I'm not sure how much of the jobs/non-jobs copypasting you've cleaned up lately, if any, since I'm way behind on reading the diffs.) That's exactly my point here. I've already cleaned up the part where the shell programs are written in jobs mode, and

Re: bmake inefficiencies

2021-01-31 Thread David Holland
On Fri, Jan 29, 2021 at 03:10:09PM -0800, Simon J. Gerraty wrote: > > It should probbaly suffice to call Job_CatchChildren after a SIGCHLD. > > It is a common misstake to assume that SIGCHLD will be receieved for each > > child - which is why looping in Job_CatchChildren is important. > > This

Re: bmake inefficiencies

2021-01-31 Thread David Holland
On Fri, Jan 29, 2021 at 11:22:40PM +0100, Roland Illig wrote: > > 1. spurious waitpid > > > > when doing buildkernel I counted the following: > > 7010 cases of error code 0 (== nothing to do) > > 371 cases of ECHILD > > 1526 cases of reaping > > I'm not familiar enough with waitpid, so I

Re: bmake inefficiencies

2021-01-29 Thread Simon J. Gerraty
Simon J. Gerraty wrote: > There's a gratuitous call to Job_CatchChildren in Job_CatchOutput. > > It should probbaly suffice to call Job_CatchChildren after a SIGCHLD. > It is a common misstake to assume that SIGCHLD will be receieved for each > child - which is why looping in Job_CatchChildren

Re: bmake inefficiencies

2021-01-29 Thread Roland Illig
On 29.01.2021 23:22, Roland Illig wrote: On 26.01.2021 15:41, Mateusz Guzik wrote: 3. avoidable temp files JobOpenTmpFile opens a file and later checks if anything was written to it. Not an actual patch, but a hack to demonstrate the fix can be found at the end. Looks like checking can be done

Re: bmake inefficiencies

2021-01-29 Thread Simon J. Gerraty
> > I had a look with truss at what bmake is doing and I think a lot of > > the syscalls can be easily avoided. Examples come from FreeBSD with > > bmake-20210110 and I don't know to what extent they are representative > > for other systems. > > > > 1. spurious waitpid > > > > when doing

Re: bmake inefficiencies

2021-01-29 Thread Roland Illig
On 26.01.2021 15:41, Mateusz Guzik wrote: I had a look with truss at what bmake is doing and I think a lot of the syscalls can be easily avoided. Examples come from FreeBSD with bmake-20210110 and I don't know to what extent they are representative for other systems. 1. spurious waitpid when

bmake inefficiencies

2021-01-26 Thread Mateusz Guzik
I had a look with truss at what bmake is doing and I think a lot of the syscalls can be easily avoided. Examples come from FreeBSD with bmake-20210110 and I don't know to what extent they are representative for other systems. 1. spurious waitpid when doing buildkernel I counted the following: