Re: How to speed up 'automake'

2022-05-23 Thread Jim Meyering
On Mon, May 23, 2022 at 1:48 PM Karl Berry wrote: > > I was going to bisect but if it doesn't fail for me in the first place... > :( > > Thanks. Indeed, reconfiguring etc. got rid of those errors. > > Now a bunch (12) of the Python tests are failing for me, presumably > because of previous Py

Re: How to speed up 'automake'

2022-05-23 Thread Karl Berry
I was going to bisect but if it doesn't fail for me in the first place... :( Thanks. Indeed, reconfiguring etc. got rid of those errors. Now a bunch (12) of the Python tests are failing for me, presumably because of previous Python changes not playing nicely with my older Python version (2.7.

Re: How to speed up 'automake'

2022-05-21 Thread Sam James
> On 22 May 2022, at 02:32, Karl Berry wrote: > > Unrelated to Jan's depend.am change, but it turns out that a previous > change broke make distcheck (768 failures). I don't feel right about > committing anything else until that is fixed. > > Error below. Jim (or anyone), can you easily advise

Re: How to speed up 'automake'

2022-05-21 Thread Karl Berry
Unrelated to Jan's depend.am change, but it turns out that a previous change broke make distcheck (768 failures). I don't feel right about committing anything else until that is fixed. Error below. Jim (or anyone), can you easily advise? I haven't delved into this part of things before. (Not sure

Re: How to speed up 'automake'

2022-05-21 Thread Karl Berry
Fancy doing it? Jim agreed :) Yeah, sorry. Other priority things have intervened :(. I have a mild hope of getting to it by tomorrow. If someone else wants to make the commit, certainly fine by me :). -k

Re: How to speed up 'automake'

2022-05-21 Thread Sam James
> On 3 May 2022, at 22:57, Karl Berry wrote: > >I see no reason why mv would be so crucial. > > Hmm, I guess you're right. Thanks for the analysis. > > The purpose of the stamp files is to avoid partial files being written > (and screwing up future makes), but if the file is zero bytes, i

Re: How to speed up 'automake'

2022-05-03 Thread Jim Meyering
On Tue, May 3, 2022 at 2:57 PM Karl Berry wrote: > > I see no reason why mv would be so crucial. > > Hmm, I guess you're right. Thanks for the analysis. > > The purpose of the stamp files is to avoid partial files being written > (and screwing up future makes), but if the file is zero bytes, i

Re: How to speed up 'automake'

2022-05-03 Thread Jim Meyering
On Tue, May 3, 2022 at 3:55 PM Nick Bowler wrote: > > On 2022-05-02, Karl Berry wrote: > > - @echo '# dummy' >$@-t && $(am__mv) $@-t $@ > > + @: >>$@ > > > > 1) does it actually speed anything up? > > The answer seems to be a resounding "yes". I tried one of my packages > on an old slow

Re: How to speed up 'automake'

2022-05-03 Thread Nick Bowler
On 2022-05-02, Karl Berry wrote: > - @echo '# dummy' >$@-t && $(am__mv) $@-t $@ > + @: >>$@ > > 1) does it actually speed anything up? The answer seems to be a resounding "yes". I tried one of my packages on an old slow PC, and changing this one line in Makefile.in cuts almost 5 seconds

Re: How to speed up 'automake'

2022-05-03 Thread Karl Berry
I see no reason why mv would be so crucial. Hmm, I guess you're right. Thanks for the analysis. The purpose of the stamp files is to avoid partial files being written (and screwing up future makes), but if the file is zero bytes, it seems that problem cannot happen. Jim, do you agree? I'll i

Re: How to speed up 'automake'

2022-05-03 Thread Jan Engelhardt
On Monday 2022-05-02 22:52, Karl Berry wrote: >- @echo '# dummy' >$@-t && $(am__mv) $@-t $@ >+ @: >>$@ > >2) without the mv I fear we are no longer noticing write failure. -k I see no reason why mv would be so crucial. Case 1. Lack of permission. The ">" operation is the one that fai

Re: How to speed up 'automake'

2022-05-02 Thread Karl Berry
- @echo '# dummy' >$@-t && $(am__mv) $@-t $@ + @: >>$@ 1) does it actually speed anything up? 2) without the mv I fear we are no longer noticing write failure. -k

Re: How to speed up 'automake'

2022-05-02 Thread Jan Engelhardt
On Monday 2022-05-02 15:31, Jan Engelhardt wrote: >> If empty files are ok (assuming they are needed at all), can they be produced >> with a minimum number of executions of a 'touch' command? > >Better yet, they can be produced with the ">>" shell builtin, skipping >spawning the touch program. (W

Re: How to speed up 'automake'

2022-05-02 Thread Thomas Jahns
On May 2, 2022, at 15:07 , Jan Engelhardt wrote: > > > On Monday 2022-05-02 14:20, Thomas Jahns wrote: > Is there a way to speed 'automake' up? [...let] ephemeral builds [..] use /dev/shm [...] >>> >>> There ought to be little difference [...] automake, that's nowhere near as >>

Re: How to speed up 'automake'

2022-05-02 Thread Jan Engelhardt
On Monday 2022-05-02 15:18, Bob Friesenhahn wrote: > On Mon, 2 May 2022, Jan Engelhardt wrote: >> >> Indeed, if a source code package consists of 1 files, then configure >> produces another 10k files for the stuff in the ".deps" directories. >> There is not much autotooling can do here, as I

Re: How to speed up 'automake'

2022-05-02 Thread Bob Friesenhahn
On Mon, 2 May 2022, Jan Engelhardt wrote: Indeed, if a source code package consists of 1 files, then configure produces another 10k files for the stuff in the ".deps" directories. There is not much autotooling can do here, as I believe pregenerating those 10k files all with "# dummy" content

Re: How to speed up 'automake'

2022-05-02 Thread Jan Engelhardt
On Monday 2022-05-02 14:20, Thomas Jahns wrote: Is there a way to speed 'automake' up? >>> >>> [...let] ephemeral builds [..] use /dev/shm [...] >> >> There ought to be little difference [...] automake, that's nowhere near as >> IO-heavy as untarring kernel source archives. It's much more

Re: How to speed up 'automake'

2022-05-02 Thread Thomas Jahns
> On Apr 30, 2022, at 01:31 , Jan Engelhardt wrote: > > On Friday 2022-04-29 22:59, Thomas Jahns wrote: >> On 4/27/22 3:49 PM, R. Diez wrote: >>> Is there a way to speed 'automake' up? >> >> While you are probably looking for system-independent advice, the best >> results >> I've had with speed

Re: How to speed up 'automake'

2022-05-01 Thread Karl Berry
automake --verbose --warnings=all --add-missing --copy Since you're talking about cron, I'm guessing the terminal output is being redirected to a file? Because if it's going to an actual tty, that can consume a surprising amount of time. Is there a way to speed 'automake' up? Only real i

Re: How to speed up 'automake'

2022-04-29 Thread Jan Engelhardt
On Friday 2022-04-29 22:59, Thomas Jahns wrote: > On 4/27/22 3:49 PM, R. Diez wrote: >> Is there a way to speed 'automake' up? > > While you are probably looking for system-independent advice, the best results > I've had with speeding up ephemeral builds is to simply use /dev/shm for > backing st

Re: How to speed up 'automake'

2022-04-29 Thread Thomas Jahns
On 4/27/22 3:49 PM, R. Diez wrote: Is there a way to speed 'automake' up? While you are probably looking for system-independent advice, the best results I've had with speeding up ephemeral builds is to simply use /dev/shm for backing storage on Linux, i.e. first try to put build directories

Re: How to speed up 'automake'

2022-04-28 Thread Markus Elfring
> That is why I am looking at speeding up the Autotools steps. > > When I run this command, I get the following trace: > > time automake --verbose --warnings=all --add-missing --copy > … > automake: thread 0: creating Makefile.in > > real    0m0,684s > user    0m0,621s > sys    0m0,085s > > It ta

How to speed up 'automake'

2022-04-27 Thread R. Diez
Hi all: I have this cross-compiling Autoconf/Automake project: https://github.com/rdiez/JtagDue/tree/master/Project This is just an example project, I have others that are bigger, and I rebuild many of them automatically overnight. That is why I am looking at speeding up the Autotools steps.