Re: release schedule for 1.9? (Was: Re: automake -vs- huge projects (1st patch))

2004-01-22 Thread Tom Tromey
 adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

adl Also, since we have switched to API-numbering, bumping that
adl version number has a cost.  For instance Debian distributes
adl automake1.4, automake1.6, automake1.7, and automake1.8.  If we
adl add another API, it'd better be worth it.

Yeah.  It turns out that Red Hat still ships 1.5 because some packages
still depend on it.  Sigh.  Obviously this doesn't scale -- at some
point it has to be so painless to upgrade that someone like Debian or
Red Hat can simply ditch 1.x and convert everything to 1.x+1 all at
once.

adl Maybe we could release an official snapshot of HEAD?  This may
adl also help to better test these uncertain subdir-suffix-rules.
adl Would that be enough?

It might.  Unfortunately I don't think we can unilaterally make a
decision like this.  We'll have to involve the other gcc maintainers.
I think the ideal for gcc is to have the entire tree requiring a
single released version of automake.  But, we'll never know if we can
do it until we try :-)

Tom




release schedule for 1.9? (Was: Re: automake -vs- huge projects (1st patch))

2004-01-20 Thread Alexandre Duret-Lutz
 Thomas == Thomas Fitzsimmons [EMAIL PROTECTED] writes:

[...]

 Thomas I was wondering about the time frame for the next
 Thomas release of automake.  Our libgcj configury upgrade
 Thomas depends on changes that are currently only available in
 Thomas automake's CVS HEAD, so we anxiously await an official
 Thomas version that includes those changes :)

No idea.

My instinct says to wait for 1.8.x to spread and stabilize.  (By
stabilize I mean that no more regression against 1.7.x are
reported.)

Also, since we have switched to API-numbering, bumping that
version number has a cost.  For instance Debian distributes
automake1.4, automake1.6, automake1.7, and automake1.8.  If we
add another API, it'd better be worth it.

Finally, the `libtool --tag' support (presently on HEAD), makes
us dependent upon the next release of Autoconf.  This is a
caching issue: Autoconf needs to know what --trace Automake will
use, so it can fill its cache beforehand.  Technically you _can_
use CVS Automake without CVS Autoconf, but it will be slower.

Maybe we could release an official snapshot of HEAD?  This may
also help to better test these uncertain subdir-suffix-rules.
Would that be enough?
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects (1st patch)

2004-01-15 Thread Thomas Fitzsimmons
On Fri, 2004-01-02 at 16:39, Tom Tromey wrote:
  Tom == Thomas Fitzsimmons [EMAIL PROTECTED] writes:
 
 [ suggestions ]
 
 Tom Anyway, this patch brings us closer to using automake-1.8 for libgcj. 
 Tom Thanks!
 
 I think all the patches are in now.  Could you try CVS automake and
 see how big the resulting Makefile.in is?
 

Hi,

I was wondering about the time frame for the next release of automake. 
Our libgcj configury upgrade depends on changes that are currently only
available in automake's CVS HEAD, so we anxiously await an official
version that includes those changes :)

Thanks,
Tom






Re: automake -vs- huge projects (1st patch)

2004-01-02 Thread Tom Tromey
 Tom == Thomas Fitzsimmons [EMAIL PROTECTED] writes:

[ suggestions ]

Tom Anyway, this patch brings us closer to using automake-1.8 for libgcj. 
Tom Thanks!

I think all the patches are in now.  Could you try CVS automake and
see how big the resulting Makefile.in is?

Tom




Re: automake -vs- huge projects

2003-12-30 Thread Tom Tromey
 adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

adl I've found this:
adl 1999-11-22  Tom Tromey  [EMAIL PROTECTED]
adl * automake.in (handle_single_transform_list): Generate explicit
adl rule for subdir objects.  Fixes new addition to subobj.test.

I looked into this a bit today.  One nice thing about having a patch
list is that it records the rationale for changes somewhere... back in
those days that sort of information was just lost.  Sigh.

I suppose the best we can hope for is to try out your change on a
different platforms and in different scenarios and hope for the
best...

Tom




mailing list archives (Was: Re: automake -vs- huge projects)

2003-12-30 Thread Alexandre Duret-Lutz
 Tom == Tom Tromey [EMAIL PROTECTED] writes:

[...]

 Tom I looked into this a bit today.  One nice thing about having a patch
 Tom list is that it records the rationale for changes somewhere... back in
 Tom those days that sort of information was just lost.  Sigh.

I thought so, until I discovered that mailing list archives at
mail.gnu.org had been truncated on 2000-09.  I have a copy of
the archives of [EMAIL PROTECTED] of 1998-04 on my disk (it
contains a lengthy discussion about --enable-maintainer-mode)
which I downloaded from mail.gnu.org and which is not available
there anymore.

Fortunately, [EMAIL PROTECTED] has been archived at several
places.  FWIW, here is where to dig copies of old mails sent to
[EMAIL PROTECTED]:

  mail.gnu.org2000-09 ... today
  www.geocrawler.com  1997... 2002
  gmane.org   2002-03 ... today
  sources.redhat.com  1999-04 ... today

For [EMAIL PROTECTED], it worse since AFAIK, it's only
archived at mail.gnu.org and gmane.org.


[EMAIL PROTECTED] is also archived on sourceware, but the
archives start at 1999-04.  I've just discovered that
http://www.geocrawler.com/lists/3/GNU/403/0/ seems to cover
[EMAIL PROTECTED] for 1997-2002.

[EMAIL PROTECTED] is only archived at mail.gnu.org and
gmane.org.  It is quite young so the mail.gnu.org archive is
complete, however I fear the day it will be cut.

[...]

-- 
Alexandre Duret-Lutz





Re: mailing list archives (Was: Re: automake -vs- huge projects)

2003-12-30 Thread Thomas Dickey
On Tue, 30 Dec 2003, Alexandre Duret-Lutz wrote:

 [EMAIL PROTECTED] is only archived at mail.gnu.org and
 gmane.org.  It is quite young so the mail.gnu.org archive is
 complete, however I fear the day it will be cut.

any idea what fraction of the archive is spam?

-- 
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net




Re: automake -vs- huge projects (1st patch)

2003-12-30 Thread Thomas Fitzsimmons
On Sat, 2003-12-27 at 21:25, Alexandre Duret-Lutz wrote:
  adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:
 
 [...]
  adl 1999-11-22  Tom Tromey  [EMAIL PROTECTED]
  adl 
  adl * automake.in (handle_single_transform_list): Generate explicit
  adl rule for subdir objects.  Fixes new addition to subobj.test.
 [...]
  adl The other side of the coin is that dependency tracking will not
  adl work anymore, because the dependency stuff for subdir/X.o should
  adl go into subdir/.deps/X.Po but the default suffix rule will put
  adl it in ./.deps/subdir/X.Po.
 [...]
  Tom Hmm, maybe this is the issue from way back.
 
  adl I'm not sure about this.  In the past dependencies were put in
  adl .deps/subdir/X.Po (i.e. where the suffix rule--at least
  adl today--would put them).  You moved them to subdir/.deps/X.Po on
  adl 2002-01-20 for PR/224.
 
 I've just noticed that the above change of 1999-11-22 occurs
 just after the merge of user-dep-gen-branch.  So perhaps you
 were right to think the issue was related to dependency
 tracking...
 
 Anyway, here is a first patch that disables explicit rules for
 subdir-objects without per-target flags.
 
 Random notes:
   - folding the depfiletmpdepfile computation into depcomp shorten
 the screen garbage resulting from the invocation of depcomp
 by one line.  This applies to all non-fastdep compilation,
 not only subdir-objects.
   - computing $depbase on-the-fly in fastdep rules adds one line
 of output, so I've arranged things so that it is only used
 when subdir-objects is set.  
   - the test suite passes with GNU Make, and I've run the 14 tests
 that uses subdir-objects with BSD Make (will start a full
 check momently).  Apart from this I have not yet tested this
 on a real project.  I'm hoping you can test this on libgcj.
   - If you omit the NEWS chunk, the patch should also apply to
 automake-1.8 vanilla.

I tested this patch with libgcj.  A test build worked fine, and
Makefile.in is *much* smaller (1.4M as opposed to 10M).  The size could
still be reduced further though.  I discussed the following with Tom on
IRC:

- all libgcj objects are listed in am__libgcj_la_SOURCES_DIST which is
included in DIST_SOURCES.  Adding the no-dist option doesn't prevent the
SOURCES_DIST and DIST_SOURCES variables from being generated, even
though they are not used anywhere.  Tom has a patch that may fix this;
I'm going to try it out.

- I've eliminated all non-essential per-target flags, but one libgcj
library still requires them.  For this library, .obj, .o and .lo targets
are generated, where .lo targets alone would suffice.

- Tom suggested that DEP_FILES may be redundant.  He suggested that
configure read the list of .Plo files from the am__include lines in
Makefile.in.

Anyway, this patch brings us closer to using automake-1.8 for libgcj. 
Thanks!

Tom






Re: automake -vs- huge projects

2003-12-30 Thread Alexandre Duret-Lutz
 Ralf == Ralf Corsepius [EMAIL PROTECTED] writes:

 Ralf On Tue, 2003-12-16 at 17:49, Tom Tromey wrote:
  The problem is, automake generates an explicit rule for each
  compilation.  Our resulting Makefile.in is nearly 9 megabytes.  This
  is really much too large -- compare to 200K with automake 1.4.

 Ralf For subdir-compilation/non-recursive Makefiles automake generates *2*
 Ralf explicit rules (cygwin and n/cygwin) for each compilation. Folding these
 Ralf two rules into one, alone would reduce the size of Makefiles using many
 Ralf files by almost factor 2.

Hmmm.  What we'd need to merge are these:

.c.o:
$(CC) -c -o $@ $
.c.obj:
$(CC) -c -o $@ `$(CYGPATH_W) $`

and these:

sub/foo.o: sub/foo.c
$(CC) -c -o $@ `test -f 'sub/foo.c' || echo '$(srcdir)/'`sub/foo.c

sub/foo.obj: sub/foo.c
$(CC) -c -o $@ `if test -f 'sub/foo.c'; then $(CYGPATH_W) 'sub/foo.c'; else 
$(CYGPATH_W) '$(srcdir)/sub/foo.c'; fi`


The extension is obviously not an issue, but how can we arrange the CYGPATH_W
stuff?   

I've thought about this:

.c.$(OBJEXT):
$(CC) -c -o $@ $(am__cpbegin)$$(am__cpend)
sub/foo.$(OBJEXT): sub/foo.c
src=`test -f 'sub/foo.c' || echo '$(srcdir)/'`sub/foo.c; \
$(CC) -c -o $@ $(am__cpbegin)$$src$(am__cpend)

and then have 
  am__cpbegin = `cygpath -w '
  am__cpend   = '`
or 
  am__cpbegin = '
  am__cpend   = '

but this is not really attractive.

A related question is why we need this `cygpath -w' stuff.
My understanding is that it is to turn special absolute
filenames such as /cygdrive/c/foo into c:\foo
(if this is true, then I think the first use of $(CYGPATH_W) in 
the sub/foo.obj rule is superfluous, since the file is relative)
so that compiler that do not understand cygwin filename can work.

Couldn't this be folded into a compiler wrapper script?
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects (1st patch)

2003-12-30 Thread Alexandre Duret-Lutz
 Thomas == Thomas Fitzsimmons [EMAIL PROTECTED] writes:

[...]

 Thomas The size could still be reduced further though.  I
 Thomas discussed the following with Tom on IRC:

 Thomas - all libgcj objects are listed in
 Thomas am__libgcj_la_SOURCES_DIST which is included in
 Thomas DIST_SOURCES.  Adding the no-dist option doesn't
 Thomas prevent the SOURCES_DIST and DIST_SOURCES variables
 Thomas from being generated, even though they are not used
 Thomas anywhere.  Tom has a patch that may fix this; I'm going
 Thomas to try it out.

 Thomas - I've eliminated all non-essential per-target flags,
 Thomas but one libgcj library still requires them.  For this
 Thomas library, .obj, .o and .lo targets are generated, where
 Thomas .lo targets alone would suffice.

 Thomas - Tom suggested that DEP_FILES may be redundant.  He
 Thomas suggested that configure read the list of .Plo files
 Thomas from the am__include lines in Makefile.in.

These sound like good ideas.  I plan to work on the .obj/.o/.lo
rules output.
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects

2003-12-30 Thread Bob Friesenhahn
On Wed, 31 Dec 2003, Alexandre Duret-Lutz wrote:

 Hmmm.  What we'd need to merge are these:

 .c.o:
   $(CC) -c -o $@ $
 .c.obj:
   $(CC) -c -o $@ `$(CYGPATH_W) $`

[stuff removed]

 The extension is obviously not an issue, but how can we arrange the CYGPATH_W
 stuff?

An simple (but ugly) approach would be to define $(CYGPATH_W) to
'echo' when not compiling under Cygwin.

 but this is not really attractive.

 A related question is why we need this `cygpath -w' stuff.
 My understanding is that it is to turn special absolute
 filenames such as /cygdrive/c/foo into c:\foo
 (if this is true, then I think the first use of $(CYGPATH_W) in
 the sub/foo.obj rule is superfluous, since the file is relative)
 so that compiler that do not understand cygwin filename can work.

It seems to me that perhaps 'CYGPATH_W' is misnamed or the use is
outdated.  Cygwin and MinGW's MSYS shell environment already
automatically translate paths.  The only case where this translation
would be necessary is for a shell environment that doesn't
automatically translate paths for native Windows binaries, or where
the automatic path translation fails.

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-18 Thread Ralf Corsepius
On Wed, 2003-12-17 at 16:01, Bob Friesenhahn wrote:
 On Wed, 17 Dec 2003, Lars Hecking wrote:
 
   What about an automake option then to generate Makefiles for GNU make?
 
 How about a new binary 'automake' program that doesn't require an
 external 'make' program at all?  It would read the Makefile.am files
 directly ...
How many times did you resort to using plain make-rule in Makefile.ams?

So, IMO, for being useful, you'd either have to extend make to accept
the *.am-syntax or to reimplement make.

Ralf






Re: automake -vs- huge projects

2003-12-18 Thread Ralf Corsepius
On Tue, 2003-12-16 at 17:49, Tom Tromey wrote:

 The problem is, automake generates an explicit rule for each
 compilation.  Our resulting Makefile.in is nearly 9 megabytes.  This
 is really much too large -- compare to 200K with automake 1.4.

For subdir-compilation/non-recursive Makefiles automake generates *2*
explicit rules (cygwin and n/cygwin) for each compilation. Folding these
two rules into one, alone would reduce the size of Makefiles using many
files by almost factor 2.

Ralf






Re: automake -vs- huge projects

2003-12-18 Thread Bob Friesenhahn
On Thu, 18 Dec 2003, Ralf Corsepius wrote:

 On Wed, 2003-12-17 at 16:01, Bob Friesenhahn wrote:
  On Wed, 17 Dec 2003, Lars Hecking wrote:
  
What about an automake option then to generate Makefiles for GNU make?
 
  How about a new binary 'automake' program that doesn't require an
  external 'make' program at all?  It would read the Makefile.am files
  directly ...
 How many times did you resort to using plain make-rule in Makefile.ams?

Plenty.

 So, IMO, for being useful, you'd either have to extend make to accept
 the *.am-syntax or to reimplement make.

Exactly.  The binary 'automake' would be a plain make which also
understands Automake syntax.

If a non-standard tool is required to be on the user's system in order
to build software, then it might as well be a tool that does
everything. :-)

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-18 Thread Paul D. Smith
%% Bob Friesenhahn [EMAIL PROTECTED] writes:

   So, IMO, for being useful, you'd either have to extend make to accept
   the *.am-syntax or to reimplement make.

  bf Exactly.  The binary 'automake' would be a plain make which also
  bf understands Automake syntax.

  bf If a non-standard tool is required to be on the user's system in order
  bf to build software, then it might as well be a tool that does
  bf everything. :-)

If you're willing to require GNU make then I'm quite confidant you could
write automake as nothing more than a suite of GNU make macros and
functions.

I doubt there would be any need for code changes to GNU make at all.

-- 
---
 Paul D. Smith [EMAIL PROTECTED]  Find some GNU make tips at:
 http://www.gnu.org  http://make.paulandlesley.org
 Please remain calm...I may be mad, but I am a professional. --Mad Scientist




Re: automake -vs- huge projects

2003-12-18 Thread Thien-Thi Nguyen
Paul D. Smith [EMAIL PROTECTED] writes:

   If you're willing to require GNU make then I'm quite confidant you
   could write automake as nothing more than a suite of GNU make macros
   and functions.

   I doubt there would be any need for code changes to GNU make at all.

i think it would be cool if automake supported GNU make specifically,
creating GNUmakefile.in from GNUmakefile.am.  a GNUmakefile.am would
imply some automake option gnu-make-only, while automake option
gnu-make would create both Makefile.in and GNUmakefile.in.  those who
use option gnu-make would need to include GNUmakefile in configure.ac
of course.

then, GNU make can DTRT since it prefers GNUmakefile over Makefile, and
other make programs can still function (unless option gnu-make-only is
used by the package maintainer in which case that should be mentioned in
README or somesuch).

thi




Re: automake -vs- huge projects

2003-12-17 Thread Alexandre Duret-Lutz
 Bob == Bob Friesenhahn [EMAIL PROTECTED] writes:

 Bob On Tue, 16 Dec 2003, Alexandre Duret-Lutz wrote:
   Bob == Bob Friesenhahn [EMAIL PROTECTED] writes:
  
 Bob Another thing that would help reduce Makefile size is to
 Bob introduce synonyms for subdirectory paths.
  
  Better: LZW compression using Makefile variables.  Any taker?  wink

 Bob Is this your way of poking fun at my idea? :-(

No, and I'm sorry it might have looked this way.  It's just a
funny idea I had when reading your mail (IOW you are the source
of inspiration and not the target).  I think I would have left
out the wink if I wanted to be ironic.
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects

2003-12-17 Thread Alexandre Duret-Lutz
 Tom == Tom Tromey [EMAIL PROTECTED] writes:

  adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:
 adl Couldn't we use the (existing) .java.o: inference rule in this
 adl case?  Actually, is there a difference between `%.o: %.java' and
 adl `.java.o:' beside portability?  -- I'm not asking about the
 adl general % construction, just about this precise case where both
 adl sides are expected to be in the same directory.

 Tom My recollection is that we tried this and ran into real problems with
 Tom it.  I don't remember what they are any more.

 Tom There's an old thread on this:

 Tom http://sources.redhat.com/ml/automake/1999-04/threads.html#00033

Thanks for the pointer!  Especially I'm glad to learn that there
are two more Make implementations where this works.

 Tom At this point I still believed that these suffix rules would work.  I
 Tom couldn't find the point where we changed things.

I've found this:

1999-11-22  Tom Tromey  [EMAIL PROTECTED]

* automake.in (handle_single_transform_list): Generate explicit
rule for subdir objects.  Fixes new addition to subobj.test.

Unfortunately the comment added to handle_single_transform_list
just says Also, if we have a subdir object, we need to generate
an explicit rule without justification, and the addition to
subobj.test is

# ... and a third bug.
grep COMPILE.*generic/a Makefile.in

A cursory scan of automake@ and bug-automake@ around that date
did not yield any insight.

 adl So the simplest part of the fix would be to disable the output
 adl of explicit rule for subdirectory sources without per-target
 adl flags when subdir-objects is used.

 Tom Yeah.  That would help us a lot, but...

 adl The other side of the coin is that dependency tracking will not
 adl work anymore, because the dependency stuff for subdir/X.o should
 adl go into subdir/.deps/X.Po but the default suffix rule will put
 adl it in ./.deps/subdir/X.Po.  I don't see an easy way to fix this,
 adl unless we add some clumsy shell computation in the suffix rules
 adl (while this can probably be folded into depcomp when it is used,
 adl it does not seem to fit well in fastdep rules).

 Tom Hmm, maybe this is the issue from way back.

I'm not sure about this.  In the past dependencies were put in
.deps/subdir/X.Po (i.e. where the suffix rule--at least
today--would put them).  You moved them to subdir/.deps/X.Po on
2002-01-20 for PR/224.

 Tom We definitely want to keep dependency tracking.  This is pretty
 Tom important.

:)

 Tom fastdep would be great, since we know we'll always be using gcc.  If
 Tom somehow dropping fastdep would get the Makefile.in to a reasonable
 Tom size, though, I'd be in favor of that.

 Tom Alternatively running sed or whatever once or twice before the
 Tom compilation isn't going to hurt as much as having a 9M Makefile.in.
 Tom So we could just do the rewrite in the suffix rule and pay the cost.

I think what to do will be clearer once someone tries to get
these things working.  I may work on this, but not before
Christmas.

 adl Note that this issue is unrelated to the %.o:%.java vs. .java.o:
 adl choice.

 Tom Not completely, since GNU make might give us handy macros that would
 Tom let us do this transformation in-line.

I confess I'm not really fond of adding a gnu-make option, since
that increases the combinatoric of all possible outputs that the
test suite ought to cover (but don't).  Sigh, if only we had a
way to generate test cases automatically...
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects

2003-12-17 Thread Lars Hecking
Paul D. Smith writes:
 %% Bob Friesenhahn [EMAIL PROTECTED] writes:
 
   bf Per-subdirectory rules and definitions can be added in order to
   bf significantly reduce the amount of redundant code, and to
   bf re-enable the capability to usefully override parts of the default
   bf Makefile.in.
 
 Not if you want to continue to generate portable makefiles.
 
 There is no way in POSIX make (for example) to generate a target in a
 subdirectory using a suffix rule.

 What about an automake option then to generate Makefiles for GNU make?





Re: automake -vs- huge projects

2003-12-17 Thread Paul D. Smith
%% Lars Hecking [EMAIL PROTECTED] writes:

  lh What about an automake option then to generate Makefiles for GNU
  lh make?

You might get slightly more takers for an option to allow use of pattern
rules, without specifically tying them to GNU make or relying on other
_very_ GNU make-centric features.

I think there are several versions of make that support pattern rules.

-- 
---
 Paul D. Smith [EMAIL PROTECTED]  Find some GNU make tips at:
 http://www.gnu.org  http://make.paulandlesley.org
 Please remain calm...I may be mad, but I am a professional. --Mad Scientist




Re: automake -vs- huge projects

2003-12-17 Thread Bob Friesenhahn
On Wed, 17 Dec 2003, Lars Hecking wrote:

  What about an automake option then to generate Makefiles for GNU make?

How about a new binary 'automake' program that doesn't require an
external 'make' program at all?  It would read the Makefile.am files
directly ...

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-17 Thread Norman Gray

Greetings,

On Wed, 17 Dec 2003, Bob Friesenhahn wrote:

 On Wed, 17 Dec 2003, Lars Hecking wrote:
 
   What about an automake option then to generate Makefiles for GNU make?
 
 How about a new binary 'automake' program that doesn't require an
 external 'make' program at all?  It would read the Makefile.am files
 directly ...

An excellent idea!

Replace `make' with a (Scheme) module for scsh[1].  That way, you get
a sane language to write rules in (unlike sh), and your makefiles are
platform independent (since scsh has been ported widely).  With that done
(and it's little more than a learning exercise), it would be conceptually
straightforward to rewrite automake as a system sitting on top of it.

Not _really_ joking,

Norman


[1] www.scsh.net

-- 
---
Norman Grayhttp://www.astro.gla.ac.uk/users/norman/
Physics and Astronomy, University of Glasgow, UK [EMAIL PROTECTED]





Re: automake -vs- huge projects

2003-12-16 Thread Bob Friesenhahn
On 16 Dec 2003, Tom Tromey wrote:

 The problem is, automake generates an explicit rule for each
 compilation.  Our resulting Makefile.in is nearly 9 megabytes.  This
 is really much too large -- compare to 200K with automake 1.4.

I have observed the same problem myself.  It is not language
dependent.  At some point (perhaps when subdirectory builds were
introduced) Automake diverged from using common rules, and started to
insert per-target rules. It seems that a bit more smarts can be used
to get close to automake 1.4 efficiency.  Per-subdirectory rules and
definitions can be added in order to significantly reduce the amount
of redundant code, and to re-enable the capability to usefully
override parts of the default Makefile.in.

 One idea we had for a fix is to introduce a new gnu-make option that
 would allow automake to generate code relying on GNU make.  Then we
 could replace all those rules with a single %.o: %.java.

This doesn't seem to be necessary.

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-16 Thread Paul D. Smith
%% Bob Friesenhahn [EMAIL PROTECTED] writes:

  bf Per-subdirectory rules and definitions can be added in order to
  bf significantly reduce the amount of redundant code, and to
  bf re-enable the capability to usefully override parts of the default
  bf Makefile.in.

Not if you want to continue to generate portable makefiles.

There is no way in POSIX make (for example) to generate a target in a
subdirectory using a suffix rule.

-- 
---
 Paul D. Smith [EMAIL PROTECTED]  Find some GNU make tips at:
 http://www.gnu.org  http://make.paulandlesley.org
 Please remain calm...I may be mad, but I am a professional. --Mad Scientist




Re: automake -vs- huge projects

2003-12-16 Thread Bob Friesenhahn
On 16 Dec 2003, Paul D. Smith wrote:

 %% Bob Friesenhahn [EMAIL PROTECTED] writes:

   bf Per-subdirectory rules and definitions can be added in order to
   bf significantly reduce the amount of redundant code, and to
   bf re-enable the capability to usefully override parts of the default
   bf Makefile.in.

 Not if you want to continue to generate portable makefiles.

 There is no way in POSIX make (for example) to generate a target in a
 subdirectory using a suffix rule.

Bummer.  I wasn't sure about that.  At the very least, an effort could
be made to decrease the Makefile size by making maximum use of macro
substitutions.

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-16 Thread Thomas Fitzsimmons
On Tue, 16 Dec 2003 09:49:44 -0700, Tom Tromey wrote:

 Tom Fitzsimmons (CCd) has been working on upgrading libgcj to use
 newer auto* tools.  This has gone swimmingly, except one problem with
 automake.
 
 A little background.  libgcj is pretty big.  It has 2,243 .java
 files at the moment.  Previously it has been using its own slightly
 hacked automake 1.4.  It used to use its own % rules to handle
 compiling .java (since 1.4 couldn't do this).  It is part of GCC,
 which recently decided as a project that requiring GNU make is ok.
 
 
 We have to use subdir-objects, both because nobody wants 2000 .o
 files in . and because we have unavoidable basename clashes between
 .java files.
 
 Also, we use a single top-level Makefile.am, as it is way more
 convenient.
 
 The problem is, automake generates an explicit rule for each
 compilation.  Our resulting Makefile.in is nearly 9 megabytes.  This
 is really much too large -- compare to 200K with automake 1.4.
 

One specifc problem I noticed is that automake is generating three
explicit rules for each target.  For example, for one .java file, these
rules are generated:

java/awt/print/Pageable.o: java/awt/print/Pageable.java
@am__fastdepGCJ_TRUE@   if $(GCJ) $(AM_GCJFLAGS) $(GCJFLAGS) -MT 
java/awt/print/Pageable.o -MD -MP -MF java/awt/print/$(DEPDIR)/Pageable.Tpo \
@am__fastdepGCJ_TRUE@ -c -o java/awt/print/Pageable.o `test -f 
'java/awt/print/Pageable.java' || echo '$(srcdir)/'`java/awt/print/Pageable.java; \
@am__fastdepGCJ_TRUE@   then mv -f java/awt/print/$(DEPDIR)/Pageable.Tpo 
java/awt/print/$(DEPDIR)/Pageable.Po; \
@am__fastdepGCJ_TRUE@   else rm -f java/awt/print/$(DEPDIR)/Pageable.Tpo; exit 1; \
@am__fastdepGCJ_TRUE@   fi
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  source='java/awt/print/Pageable.java' 
object='java/awt/print/Pageable.o' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  depfile='java/awt/print/$(DEPDIR)/Pageable.Po' 
tmpdepfile='java/awt/print/$(DEPDIR)/Pageable.TPo' @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  $(GCJDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepGCJ_FALSE@  $(GCJ) $(AM_GCJFLAGS) $(GCJFLAGS) -c -o 
java/awt/print/Pageable.o `test -f 'java/awt/print/Pageable.java' || echo 
'$(srcdir)/'`java/awt/print/Pageable.java

java/awt/print/Pageable.obj: java/awt/print/Pageable.java
@am__fastdepGCJ_TRUE@   if $(GCJ) $(AM_GCJFLAGS) $(GCJFLAGS) -MT 
java/awt/print/Pageable.obj -MD -MP -MF java/awt/print/$(DEPDIR)/Pageable.Tpo \
@am__fastdepGCJ_TRUE@ -c -o java/awt/print/Pageable.obj `if test -f 
'java/awt/print/Pageable.java'; then $(CYGPATH_W) 'java/awt/print/Pageable.java'; else 
$(CYGPATH_W) '$(srcdir)/java/awt/print/Pageable.java'; fi`; \
@am__fastdepGCJ_TRUE@   then mv -f java/awt/print/$(DEPDIR)/Pageable.Tpo 
java/awt/print/$(DEPDIR)/Pageable.Po; \
@am__fastdepGCJ_TRUE@   else rm -f java/awt/print/$(DEPDIR)/Pageable.Tpo; exit 1; \
@am__fastdepGCJ_TRUE@   fi
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  source='java/awt/print/Pageable.java' 
object='java/awt/print/Pageable.obj' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  depfile='java/awt/print/$(DEPDIR)/Pageable.Po' 
tmpdepfile='java/awt/print/$(DEPDIR)/Pageable.TPo' @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  $(GCJDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepGCJ_FALSE@  $(GCJ) $(AM_GCJFLAGS) $(GCJFLAGS) -c -o 
java/awt/print/Pageable.obj `if test -f 'java/awt/print/Pageable.java'; then 
$(CYGPATH_W) 'java/awt/print/Pageable.java'; else $(CYGPATH_W) 
'$(srcdir)/java/awt/print/Pageable.java'; fi`

java/awt/print/Pageable.lo: java/awt/print/Pageable.java
@am__fastdepGCJ_TRUE@   if $(LIBTOOL) --mode=compile $(GCJ) $(AM_GCJFLAGS) $(GCJFLAGS) 
-MT java/awt/print/Pageable.lo -MD -MP -MF java/awt/print/$(DEPDIR)/Pageable.Tpo \
@am__fastdepGCJ_TRUE@ -c -o java/awt/print/Pageable.lo `test -f 
'java/awt/print/Pageable.java' || echo '$(srcdir)/'`java/awt/print/Pageable.java; \
@am__fastdepGCJ_TRUE@   then mv -f java/awt/print/$(DEPDIR)/Pageable.Tpo 
java/awt/print/$(DEPDIR)/Pageable.Plo; \
@am__fastdepGCJ_TRUE@   else rm -f java/awt/print/$(DEPDIR)/Pageable.Tpo; exit 1; \
@am__fastdepGCJ_TRUE@   fi
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  source='java/awt/print/Pageable.java' 
object='java/awt/print/Pageable.lo' libtool=yes @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  
depfile='java/awt/print/$(DEPDIR)/Pageable.Plo' 
tmpdepfile='java/awt/print/$(DEPDIR)/Pageable.TPlo' @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepGCJ_FALSE@  $(GCJDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepGCJ_FALSE@  $(LIBTOOL) --mode=compile $(GCJ) $(AM_GCJFLAGS) $(GCJFLAGS) -c 
-o java/awt/print/Pageable.lo `test -f 'java/awt/print/Pageable.java' || echo 
'$(srcdir)/'`java/awt/print/Pageable.java

I would only expect rules for .lo files, since we are generating libtool
libraries.  Getting rid of the other targets would cut out *a lot* of the
bulk.

Tom






Re: automake -vs- huge projects

2003-12-16 Thread Bob Friesenhahn
Another thing that would help reduce Makefile size is to introduce
synonyms for subdirectory paths.  For example,

  java/awt/print/$(DEPDIR)/

could be reduced to $(am_f123).  This would of course obfusticate the
Makefile, so it should be made optional, or allow the user to specify
part of the name so it is still comprehensible.

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-16 Thread Bob Friesenhahn
I would like to point out that my previous gripe about the unnecessary
translation of Automake target names (particularly when the target
includes a path), and the subsequent proposal to add user-level target
synonyms, dovetails nicely with the need to reduce the Makefile size.
The synonyms can help eliminate the size increase caused by the
directory path itself.  For typical subdirectory path lengths, this
could result in a 40% Makefile size reduction.

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-16 Thread Alexandre Duret-Lutz
 Tom == Tom Tromey [EMAIL PROTECTED] writes:

[...]
 Tom We have to use subdir-objects
[...]
 Tom Also, we use a single top-level Makefile.am
[...]
 Tom The problem is, automake generates an explicit rule for each
 Tom compilation.  Our resulting Makefile.in is nearly 9 megabytes.  This
 Tom is really much too large -- compare to 200K with automake 1.4.

 Tom One idea we had for a fix is to introduce a new gnu-make option that
 Tom would allow automake to generate code relying on GNU make.  Then we
 Tom could replace all those rules with a single %.o: %.java.

Couldn't we use the (existing) .java.o: inference rule in this
case?  Actually, is there a difference between `%.o: %.java' and
`.java.o:' beside portability?  -- I'm not asking about the
general % construction, just about this precise case where both
sides are expected to be in the same directory.

I've done some limited testing (with GNU make, BSD make,
OSF1/Tru64 Make, and HP-UX make) and .src.dest: rules appear to
work for files in subdirectory even in VPATH configurations.  I
wonder if there are Make implementations where this does not
work.

So the simplest part of the fix would be to disable the output
of explicit rule for subdirectory sources without per-target
flags when subdir-objects is used.

The other side of the coin is that dependency tracking will not
work anymore, because the dependency stuff for subdir/X.o should
go into subdir/.deps/X.Po but the default suffix rule will put
it in ./.deps/subdir/X.Po.  I don't see an easy way to fix this,
unless we add some clumsy shell computation in the suffix rules
(while this can probably be folded into depcomp when it is used,
it does not seem to fit well in fastdep rules).  Note that this
issue is unrelated to the %.o:%.java vs. .java.o: choice.

[...]

-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects

2003-12-16 Thread Alexandre Duret-Lutz
 Bob == Bob Friesenhahn [EMAIL PROTECTED] writes:

 Bob Another thing that would help reduce Makefile size is to
 Bob introduce synonyms for subdirectory paths.

Better: LZW compression using Makefile variables.  Any taker?  wink
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects

2003-12-16 Thread Bob Friesenhahn
On Tue, 16 Dec 2003, Alexandre Duret-Lutz wrote:

  Bob == Bob Friesenhahn [EMAIL PROTECTED] writes:

  Bob Another thing that would help reduce Makefile size is to
  Bob introduce synonyms for subdirectory paths.

 Better: LZW compression using Makefile variables.  Any taker?  wink

Is this your way of poking fun at my idea? :-(

Bob
==
Bob Friesenhahn
[EMAIL PROTECTED]
http://www.simplesystems.org/users/bfriesen





Re: automake -vs- huge projects

2003-12-16 Thread Alexandre Duret-Lutz
 Thomas == Thomas Fitzsimmons [EMAIL PROTECTED] writes:

[...]
 Thomas One specifc problem I noticed is that automake is
 Thomas generating three explicit rules for each target.  For
 Thomas example, for one .java file, these rules are generated:
[...]
 Thomas I would only expect rules for .lo files, since we are
 Thomas generating libtool libraries.  Getting rid of the other
 Thomas targets would cut out *a lot* of the bulk.

Indeed!

I think we can construct cases that will break if any of the .o
or .lo rules are left out, for instance

  bin_PROGRAMS = foo
  foo_SOURCES = a.c b.c
  foo_LDADD = @MOREFOO@
  lib_LTLIBRARIES = libbar.la
  libbar_la_SOURCES = 1.c 2.c
  libbar_la_LIBADD = @MOREBAR@

now think about @MOREFOO@ being substituted into 1.o or 2.o, and
@MOREBAR@ being replaced by a.la or b.la...  However it seems
fair to require that the above snippet be rewritten as

  bin_PROGRAMS = foo
  foo_SOURCES = a.c b.c
  foo_LDADD = @MOREFOO@
  EXTRA_foo_SOURCES = 1.c 2.c
  lib_LTLIBRARIES = libbar.la
  libbar_la_SOURCES = 1.c 2.c
  libbar_la_LDADD =  @MOREBAR@
  EXTRA_libbar_la_SOURCES = a.c b.c

like the manual says (oh well, I hope it says so).  And in this
second example it is clear which rules Automake needs to output.

Furthermore, generally it does not work to compile both the .o
and .lo objects of a source file (in the last example Automake
is expected to warn that these files are being built both with
and without Libtool), so it sounds safe to remove the rules
which are not used.
-- 
Alexandre Duret-Lutz





Re: automake -vs- huge projects

2003-12-16 Thread Tom Tromey
 adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

adl Couldn't we use the (existing) .java.o: inference rule in this
adl case?  Actually, is there a difference between `%.o: %.java' and
adl `.java.o:' beside portability?  -- I'm not asking about the
adl general % construction, just about this precise case where both
adl sides are expected to be in the same directory.

My recollection is that we tried this and ran into real problems with
it.  I don't remember what they are any more.

There's an old thread on this:

  http://sources.redhat.com/ml/automake/1999-04/threads.html#00033

At this point I still believed that these suffix rules would work.  I
couldn't find the point where we changed things.

adl So the simplest part of the fix would be to disable the output
adl of explicit rule for subdirectory sources without per-target
adl flags when subdir-objects is used.

Yeah.  That would help us a lot, but...

adl The other side of the coin is that dependency tracking will not
adl work anymore, because the dependency stuff for subdir/X.o should
adl go into subdir/.deps/X.Po but the default suffix rule will put
adl it in ./.deps/subdir/X.Po.  I don't see an easy way to fix this,
adl unless we add some clumsy shell computation in the suffix rules
adl (while this can probably be folded into depcomp when it is used,
adl it does not seem to fit well in fastdep rules).

Hmm, maybe this is the issue from way back.

We definitely want to keep dependency tracking.  This is pretty
important.

fastdep would be great, since we know we'll always be using gcc.  If
somehow dropping fastdep would get the Makefile.in to a reasonable
size, though, I'd be in favor of that.

Alternatively running sed or whatever once or twice before the
compilation isn't going to hurt as much as having a 9M Makefile.in.
So we could just do the rewrite in the suffix rule and pay the cost.

adl Note that this issue is unrelated to the %.o:%.java vs. .java.o:
adl choice.

Not completely, since GNU make might give us handy macros that would
let us do this transformation in-line.

Tom




Re: automake -vs- huge projects

2003-12-16 Thread Tom Tromey
 adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes:

adl Furthermore, generally it does not work to compile both the .o
adl and .lo objects of a source file (in the last example Automake
adl is expected to warn that these files are being built both with
adl and without Libtool), so it sounds safe to remove the rules
adl which are not used.

This would be a great change, but unfortunately it only gets us down
to 3M, which is still about 6x too large.

Tom