Re: [GSoC] Early design discussion for TAP/SubUnit support in automake.

[Robert Collins]

On Thu, Jul 7, 2011 at 7:03 AM, Stefano Lattarini
 wrote:
> Hello Robert.
>> >>
>> > OTOH, I do believe this is a real concern, to be carefully addressed and
>> > tested for.  Thanks for bringing this up.
>>
>> For Both TAP and subunit the test script running needs to feed into a
>> single parser:
>>
> This is not possible with the current implementation/design of third-party
> test drivers in Automake.  Every test script listed in $(TESTS) is passed
> *separately* to dedicated instance of the proper test driver; this means
> that the output of *each* test script in $(TESTS) will be parsed by a
> different instance of the TAP (or SubUnit) parser.

Crossed wires: I meant each test script having a different instance: I
phrased it poorly. Put differently there should be a 1:1 mapping
between test script that runs and parser instances.

-Rob

Re: [GSoC] Early design discussion for TAP/SubUnit support in automake.

[Robert Collins]

Very sorry for the slow response, been EBUSY with real-life.

On Sun, May 22, 2011 at 11:42 PM, Stefano Lattarini
 wrote:
> On Sunday 22 May 2011, Ralf Wildenhues wrote:
>> Hi Stefano, and sorry for the long delay,
>>
> No problem, you had warned me in due time about such possible delays this
> month; so there's really no need to apologize.
>
>> * Stefano Lattarini wrote on Fri, Apr 29, 2011 at 11:21:06AM CEST:
>> > Now that my GSoC application "automake - Interfacing with a test protocol
>> > like TAP or subunit" has been officially accepted, I'd like to start
>> > discussing with the community some early, high-level design and interface
>> > decisions.
>>
>>
>> >  1. Reuse parallel-tests "framework"
>> >  ---
>> >
>> >   The new TAP/SubUnit support should reuse as much of the current
>> >   parallel-tests implementation and semantics as possible.  In particular,
>> >   it should be able to run different test scripts in parallel, generate a
>> >   `.log' file for each test script and a "summarizing" `test-suite.log'
>> >   file, honour the make variables AM_TESTS_ENVIRONMENT, TESTS_ENVIRONMENT
>> >   and AM_COLOR_TESTS and the environment variable VERBOSE, and support
>> >   different extensions for the test scripts, with extension-specific "log
>> >   compilers" and flags (the stuff enabled by TESTS_EXTENSIONS,
>> >   _LOG_COMPILER, etc.).
>>
>> Sounds all sane.
>>
>> >   The XFAIL_TESTS variable might be still supported for the sake of
>> >   backward-compatibility (see below for the details), but it should be
>> >   deprecated, since TAP and SubUnit offer better and more granular ways
>> >   to express expected failures.
>>
>> OK.
>>
>> In another mail:
>> > Thinking again about this, it might be worth trying to be even more 
>> > consistent
>> > with the existing parallel-tests functionality, and use an 
>> > `ext_TEST_PROTOCOL'
>> > variable (or similar) instead of a global `tests-protocol' option.  With 
>> > some
>> > tweaking to the post-processing of `.log' files done in `lib/am/check.am' 
>> > (to
>> > generate `$(TEST_SUITE_LOG)'), this might allow greater code reuse and a 
>> > more
>> > consistent API.
>> >
>> > I've started experimenting with this idea, and I'm not seeing any obvious
>> > shortcoming right now.  I'm hoping I'll be able to post some experimental
>> > patches soon enough.
>>
>> Allowing to specify that per-test is a good idea for transitioning test
>> suites.
>>
> About this, in my first two "tentative" patches:
>  
> I've taken an even more general approach, allowing the developer to define
> and use his own program(s) to:
>  1. launch his test scripts,
>  2. interpreter their results,
>  3. display these results on screen, and
>  4. format and generate the log files.
> All of this is attainable simply by assigning a variable `LOG_WRAPPER'
> (and extensione-specific counterparts of it), and, well, obviously
> providing a real "driver" script that obeys to minimal command-line
> interface (so that it can grasp the options the Automake-generated
> Makefiles passes to it).  Then we will hopefully be able to implement
> our TAP/SubUnit parsers on the top of this feature (thus making it
> indirectly more tested, which is always good for a new feature).

If the subunit parser just gets all the output from the test script,
you might want to use the subunit parser itself: folk in the
bootstrap-set couldn't (obviously), but most other environments can
use high level languages freely. (I don't object to more parser
implementations existing, I'm just thinking about reuse where
possible.

>> I hope to look into the posted patches later today.
>>
> About this, please note that I might be AFK until this evening.  So have
> no haste.
>
>> >  2. New automake option `tests-protocol'
>> >  ---
>> >
>> >  The Tap/SubUnit support in the Automake-generated testsuite drivers
>> >  should be enabled by a new (argument-requiring) option `tests-protocol',
>> >  that will be used to specify the level of support for, detection of, and
>> >  enforcing of SubUnit/TAP streams.
>> >
>> >  The possible values for `tests-protocol' will be:
>> >   - tests-protocol=tap
>> >     All test scripts are expected to use the TAP protocol.
>> >   - tests-protocol=subunit
>> >     All test scripts are expected to use the SubUnit protocol.
>> >   - tests-protocol=adaptive
>>
>> The way you describe "adaptive", it sounds like it should rather be
>> named something like "detect" or "detected" or so.
>>
> I'd like to withdraw this proposal now that we can define per-extension test
> protocols.  Having our hypotetical "client developer" rename his test scripts
> as they get converted to use TAP/SubUnit is IMO better than we having to
> implement in Automake a probably non-trivial "metaparser" that could end up
> being scarcerly used anyway.  WDYT?
>
>> >     Each test script is expect

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Robert Collins]

On Tue, Mar 22, 2011 at 8:41 AM, Stefano Lattarini
> to its suboptimal documentation.  So I'm going to ask: Robert, as
> the main proposer/supporter of the SubUnit protocol here, would you
> be willing and ready to help me out during my prospective work with
> GSoC, if I update my application's goal to read "Support SubUnit
> (and also TAP as derivation) in Automake-generated testsuites"?
> Maybe we could also improve SubUnit's documentation along the way,
> which would help both me and Automake, and also improve SubUnit
> itself and make it more "palatable" for potential adopters.

I'll be delighted to help by clarifying things or making sensible
small improvements. We should structure things so there is a clear API
between whatever automake needs and subunit providing it (this allows
subunit to continue to improve without automake needing to change
futher, and vice versa). I would offer to make large changes if
needed, but realistically, work is in the middle of a crucial project
that is going to be going on for ~ 3-4 more months, so I am somewhat
time constrained :). That said, Jelmer Vernooij may well - he has
written a number of Perl subunit tools in the samba project, and they
are likely to be at least somewhat relevant to whatever automake needs
are here.

-Rob

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Robert Collins]

On Sun, Mar 20, 2011 at 10:01 PM, Stefano Lattarini
 wrote:
> Hello Robert, and thanks for the feedback.
>
> On Sunday 20 March 2011, Robert Collins wrote:
>> On Sat, Mar 19, 2011 at 1:03 PM, Stefano Lattarini
>>  wrote:
>> > ABSTRACT:
>> >
>> >  The Test Anything Protocol (TAP) is a simple text-based protocol
>> >  that allows communication between test scripts and a test harness.
>> ...
>> > Now, in all honesty, I must say that I've chosen TAP not only for its
>> > objective qualities and merits, but also because I have some previous
>> > knowledge of it (which allowed me to present a more concrete proposal
>> > and a meaningful roadmap) and I personally like it (which will probably
>> > be a powerful motivator to overcome the unavoidable hurdles I'll
>> > encounter down the road).
>> >
>> > Still, there might be very valid competing alternatives to TAP out there,
>> > which I might not know about, but that in the long run would offer
>> > Automake more advantages and interoperabilty, thus outweighting the two
>> > "personal" advantages of TAP I've reported above.  So, if anyone who's
>> > reading this has    proposals about viable alternatives to TAP, please
>> > speak up -- your contribution is appreciated!
>>
>> TAP is an extremely simple protocol, and the extensions to it to
>> support things like not needing to maintain the count of tests,
>> additional debug data and so on are pretty rudimentary. subunit, which
>> I've mentioned before was written after TAP, to solve similar problems
>> and address the issues in TAP itself.
>>
>> While the users of a protocol aren't really an indicator of the
>> protocols worth, projects as large as samba are using subunit.
>>
>> Unlike TAP subunit supports attachments (binary and text) to tests,
>> test naming, tagging, timestamping (permitting robust timing data even
>> in parallelised or distributed testing).
>>
> Is there some document that describes the subunit features in depth *and*
> with examples?  I mean something similar to what the following is for TAP:
>  <http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod>
> That would be very useful to do a proper comparison, and weight advantages
> and disadvatnages.

http://bazaar.launchpad.net/~subunit/subunit/trunk/view/head:/README#L105 and
http://bazaar.launchpad.net/~subunit/subunit/trunk/view/head:/README#L134
may cover what you're asking.

>> Integrating with TAP is basically uninteresting to anyone working in a
>> high level language: Python, Ruby,Java, C++ etc.
>>
> How so? (honest question)

Python tests are most often written using one of:
 - unittest (built in)
 - doctest (built in, horrible ;))
 - py.test (external, quite popular)
 - nose/trial/zope.testing/any of a number of other third party
extensions that use the basic unittest object model: that an xUnit
model,  and TAP does a very poor job of interfaces with xUnit runners.
 - there are some other custom test runners around, including the TAP
one, but they (IME) have nearly-no uptake.

Java has the same xUnit test runner model, and it dominates the
testing frameworks there - massively and comprehensively. All the
domain specific, literate testing, BDD testing frameworks I've seen
are built to be compatible with junit.

Ruby seems to be dominated by rspec and cucumber, both of which are
entire runners in their own right.

C++ has cppunit and a recent addition, the google test runner, as its
dominate test environments. CPPUnit is another xUnit framework, as is
googletest.

There are four primary targets for test run integration:
 - local IDE : tests can drop straight into a debugger : things like
TAP and subunit generally just get in the way.
 - local via a subprocess : subunit/TAP style filtering and chain can
be useful to get effective output
 - distributor / end user builds : report success/failure and provide
a diagnostic to return to the developer
 - automated test runs - Jenkins/Buildbot/parallel test farms :
provide effective introspection of all test run, and as much
environmental data as possible to allow effective reproduction and
correction

The overly simple model of TAP does not fit well with the xUnit style
of tests - and its this friction that to me explains why TAP really
seems isolated to the Perl community (where it is extremely popular).

-Rob

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Robert Collins]

On Sun, Mar 20, 2011 at 8:53 PM, Ralf Wildenhues  wrote:
> Hi Robert,
>
> thanks for the feedback.  I have a couple of questions:
>
> * Robert Collins wrote on Sun, Mar 20, 2011 at 05:10:16AM CET:
>> TAP is an extremely simple protocol, and the extensions to it to
>> support things like not needing to maintain the count of tests,
>> additional debug data and so on are pretty rudimentary. subunit, which
>> I've mentioned before was written after TAP, to solve similar problems
>> and address the issues in TAP itself.
>
> Are TAP and subunit compatible on their common subset?  If not, why not?

You can convert TAP to subunit, and you can convert the things TAP can
represent from subunit to TAP. subunit's core is more structured than
TAP, so the two protocols don't pun as each other at all.

>> Unlike TAP subunit supports attachments (binary and text) to tests,
>
> Handling of binary data may end up being quite tricky within a
> restricted Posix environment with only a few tools available.
> What if a consumer cannot handle them?  Is there possibility
> for graceful fallback?

I didn't think posix had 7 bit limits? Anyhow, there is a baseline
profile which assumes just a single description of the error in a test
- it uses \r] to delimit a traceback. Alternatively, a C parser - on
my 'sometime' TODO list - will probably clock in small enough to
bundle for projects very low in the dependency stack.

I think a reasonable approach would be:
 - have a tiny minimal 'list test and report' consumer which can be
bundled for the bootstrap closure of projects.
 - detect (via autoconf) the presence of existing subunit consumers
and preferentially use those.

However, even this may not be needed- most bring-up situations involve
cross compiling these days, so just depending on the presence of an
executable subunit-stats (for instance) may be sufficient : cross
compile a set of C based subunit consumers, done.

>> test naming, tagging, timestamping (permitting robust timing data even
>> in parallelised or distributed testing).
>>
>> Integrating with TAP is basically uninteresting to anyone working in a
>> high level language: Python, Ruby,Java, C++ etc.
>
> Well, Perl is a high-level language too, I'd say.  ;-)

It is and it isn't - it has a very-close-to-the-metal feel to it. perl
6 may change this more dramatically :).

-Rob

Re: [GSoC Proposal] automake - Interfacing with a test protocol like TAP or subunit

[Robert Collins]

On Sat, Mar 19, 2011 at 1:03 PM, Stefano Lattarini
 wrote:
> ABSTRACT:
>
>  The Test Anything Protocol (TAP) is a simple text-based protocol
>  that allows communication between test scripts and a test harness.
...
> Now, in all honesty, I must say that I've chosen TAP not only for its
> objective qualities and merits, but also because I have some previous
> knowledge of it (which allowed me to present a more concrete proposal
> and a meaningful roadmap) and I personally like it (which will probably
> be a powerful motivator to overcome the unavoidable hurdles I'll
> encounter down the road).
>
> Still, there might be very valid competing alternatives to TAP out there,
> which I might not know about, but that in the long run would offer
> Automake more advantages and interoperabilty, thus outweighting the two
> "personal" advantages of TAP I've reported above.  So, if anyone who's
> reading this has    proposals about viable alternatives to TAP, please
> speak up -- your contribution is appreciated!

TAP is an extremely simple protocol, and the extensions to it to
support things like not needing to maintain the count of tests,
additional debug data and so on are pretty rudimentary. subunit, which
I've mentioned before was written after TAP, to solve similar problems
and address the issues in TAP itself.

While the users of a protocol aren't really an indicator of the
protocols worth, projects as large as samba are using subunit.

Unlike TAP subunit supports attachments (binary and text) to tests,
test naming, tagging, timestamping (permitting robust timing data even
in parallelised or distributed testing).

Integrating with TAP is basically uninteresting to anyone working in a
high level language: Python, Ruby,Java, C++ etc.

-Rob

Re: Autotools GSoC ideas

[Robert Collins]

On Wed, Mar 9, 2011 at 8:39 AM, Stefano Lattarini
 wrote:
> I don't know how the GSoC proposals are evaluated, but if reviewers tend
> to prefer more precise goals, extending the proposal in this way might
> not be a smart move.  Maybe something like the following would be better?
>
>  ``Interfacing with the Test Anything Protocol (TAP).  If possible, try
>   to write an implementation that will allow future extensions to
>   similar but more advanced advanced protocols (e.g., subunit, which
>   is similar to TAP but slightly more structured, capable of handling
>   binary attachments, and so on).''

You could - or you could just write to the most capable and let folk
insert a filter (e.g. tap2subunit, included in the subunit package) if
they are using a different protocol themselves.

There are a whole bunch of such protocols with varying capabilities
around - tap, subunit, junit's xml format, glib's xml format, at least
one json based format...

-Rob

Re: Autotools GSoC ideas

[Robert Collins]

On Tue, Mar 8, 2011 at 1:10 AM, Ralf Wildenhues  wrote:
> Hello,
>
> I'll throw a couple of suggestions for Autotools out there:
>
> 1) Interfacing with the Test Anything Protocol (TAP) (or maybe another
> test protocol?).  Automake-generated Makefiles could be consumers of the
> protocol for third-party testsuites.  Automake-generated simple
> testsuites (TESTS) and Autoconf-generated Autotest testsuites could be
> producers.
>
> This project would entail both investigating the possible advantages
> of either step, prototyping an implementation, and getting it ready
> to merge into the code.  Each step on its own would already be useful
> on its own, and small enough to be completed within the given time
> frame.

I've mentioned subunit before - its similar to TAP but slightly more
structured and capable of handling binary attachments and so on.

-Rob

Re: --gnits, AC_INIT and VERSION

[Robert Collins]

On Sun, 2009-11-29 at 22:10 +0100, Ralf Wildenhues wrote:
> Hello Robert, and sorry for not replying on this earlier:

Hi - no problems ;).

> * Robert Collins wrote on Wed, Sep 23, 2009 at 10:03:42AM CEST:
> > There was discussion about getting version numbers from VCS recently;
> > I've done a slightly different thing for a while now:
> > AC_DEFUN([SUBUNIT_MAJOR_VERSION], [0])
> > AC_DEFUN([SUBUNIT_MINOR_VERSION], [0])
> > AC_DEFUN([SUBUNIT_MICRO_VERSION], [2])
> > AC_DEFUN([SUBUNIT_VERSION],
> > [SUBUNIT_MAJOR_VERSION.SUBUNIT_MINOR_VERSION.SUBUNIT_MICRO_VERSION])
> > AC_PREREQ([2.59])
> > AC_INIT([subunit], [SUBUNIT_VERSION], [contact...])
> > AC_SUBST([SUBUNIT_MAJOR_VERSION]) ...
> > 
> > This works well, until gnits is added to AM_INIT_AUTOMAKE.
> > 
> > It looks like it expects a literal, not a macro in AC_INIT ?
> 
> Yeah, the test unfortunately needs the version number literally at m4
> time; you can achieve that with something like
> 
> m4_define([SUBUNIT_MAJOR_VERSION], [0])
> m4_define([SUBUNIT_MINOR_VERSION], [0])
> m4_define([SUBUNIT_MICRO_VERSION], [2])
> m4_define([SUBUNIT_VERSION],
>  
> m4_defn([SUBUNIT_MAJOR_VERSION]).m4_defn([SUBUNIT_MINOR_VERSION]).m4_defn([SUBUNIT_MICRO_VERSION]))
> AC_PREREQ([2.59])
> AC_INIT([subunit], m4_defn([SUBUNIT_VERSION]), [contact...])
> 
> BTW, and this is why I'm adding bug-autoconf here: Not changing the
> AC_DEFUNs here to m4_define causes Autoconf 2.65 to barf with
>   configure.ac:7: error: m4_divert_push: cannot change diversion to `GROW' 
> inside m4_expand
> 
> which seems a bit surprising to me.  What's the NEWS entry that
> described this change in behavior?  Or was that always problematic
> and we only didn't notice earlier by chance?

I don't recall seeing a NEWS entry relating to this; what happened is
that I was adding this idiom to an existing gnits project and it failed.
We could bisect for it I guess, but I suspect its latent for a while -
and will likely raise trouble for folk wanting to make their version
numbers very dynamic too.

-Rob


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive automake

[Robert Collins]

On Sun, 2009-10-18 at 08:39 +0200, Ralf Wildenhues wrote:

> > http://sources.redhat.com/ml/automake/2001-08/msg00112.html
> > 
> > This added a new directive 'subdir_include' which does an include but
> > adjusts all the paths in the make/automake rules in the included
> > fragment to the relative path to the included rules.
> 
> The devil is in the details.  What about -I paths in *_CPPFLAGS?  What
> with substituted variables?  What about rewritten variable names, such
> as: libfoo_la_SOURCES becomes sub_libfoo_la_SOURCES, and what if the
> user references $(libfoo_la_SOURCES) elsewhere, say, in
> libbar_la_SOURCES?
> 
> No.  Search for several prior discussions on the Automake lists for why
> this cannot be done safely without highly altering the set of allowed
> semantics, and things the user can expect.

I'll take it on faith; I must have missed those discussions (there was a
period while I didn't receive forwarded mail from my old cygwin address
before I resubscribed). Regardless, if something usable is added, +1.

-Rob


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive automake

[Robert Collins]

On Sat, 2009-10-17 at 20:09 -0500, Bob Friesenhahn wrote:
> 
> I complained about this perhaps five years ago since it is the most 
> annoying issue related to non-recursive build.  There was some 
> discussion on this list at that time but nothing was done to make 
> things better.
> 
> It seems that a problem is that much of the Makefile.am file is
> simply 
> copied to the output Makefile.in and so these parts would need to be 
> re-written rather than copied.  The good news is that perl is good at 
> re-writing text. 

The way I tackled this in my proof of concept in 2001 was via a
rewriting include:

http://sources.redhat.com/ml/automake/2001-08/msg00112.html

This added a new directive 'subdir_include' which does an include but
adjusts all the paths in the make/automake rules in the included
fragment to the relative path to the included rules.

e.g. subdir_include foo/Makefile.am
would prepend 'foo' to the paths in foo/Makefile.am.

Automake's core has probably changed so much that the patch is not worth
even reading, but the concept worked tolerably well ;).

-Rob


signature.asc
Description: This is a digitally signed message part

Re: how to detect broken install-sh?

[Robert Collins]

On Mon, 2009-09-28 at 08:56 +0200, Ralf Wildenhues wrote:
> 
> You're much better off arguing that packages update to Autoconf 2.64,
> in many cases the configure script will shrink by more than 15K over
> the one generated by 2.63 (and it'll be a bit faster, too). 

Nice! - and I think they should - the two positions are very compatible.

-Rob


signature.asc
Description: This is a digitally signed message part

Re: how to detect broken install-sh?

[Robert Collins]

On Sun, 2009-09-27 at 20:38 -0500, Bob Friesenhahn wrote:

> > Thats the key number - the amount of benefit that install-sh gives you.
> 
> This violates a core principle of GNU in that "benefits" should be for 
> the benefit of the recipients of the software rather than for the for 
> the developers of it.  GNU is a communistic/Marxist type model rather 
> than a capitalistic model.  In the old days, the benefits were for the 
> developers and the users had to muddle through a difficult procedure 
> for every package that they installed.

I meant the benefit to the community, or even to the folk that end up
needing install-sh. I think portability is a great thing, but I also
think repeatedly solving the same problem isn't: particular when bug
fixes exist :). Anyhow, we're way off the original topic here, and I've
achieved my goal - to put my toe in the water about this sort of
change ;).

> To be sure, I will be quite supportive of a build framework if it is 
> based on a small package which is easily installed, and the build no 
> longer needs to be cobbled together with a mismash of Unix utilities. 
> Of course this build environment needs to be self-contained, well 
> supported, and would probably take five or seven years to fully 
> develop.  There have been a number of independent attempts in this 
> direction but it seems that none has come close to the popularity of 
> autotools.

All the ones I've seen have been 90% (or less) solutions and have often
[but not always] decided to replace Make with something less powerful :
a mistake IMO. I'm fairly sure I know what it would take to do a 100%
solution, but its daunting ;). I'm thinking of cmake, waf, scons,
primarily here, with cook, bake and others coming in as less well known
stabs in the same direction.

-Rob


signature.asc
Description: This is a digitally signed message part

Re: how to detect broken install-sh?

[Robert Collins]

On Sun, 2009-09-27 at 18:59 -0500, Bob Friesenhahn wrote:
> On Mon, 28 Sep 2009, Robert Collins wrote:
> >
> > The landscape has changed though, and I suspect that if we gather stats
> > about this we'll see that install-sh is dead weight for most packages
> > nearly all of the time.
> 
> Maybe the landscape has changed for you, but not necessarily for 
> everyone.  Installing "coreutils" could be quite a burden and the 
> tools might conflict with the OS-provided equivalents.

I'm not a strong enough believer in the Copenhagen school to think that
I'm in a different universe. I'll agree that the distribution of OSs is
different for each open source project. But - data needed - for either
of us to reason effectively on this. As far as conflicting, there are
multiple well established places to install things that won't
conflict: /opt /usr/local ~/local - plus you can just make one up and
put it in your path.

> > Its true that it is not a lot of dead weight, but at some point we
> > should be raising the bar - ever so slightly - on what we bundle into
> > the tarball. At one point we never required a Make implementation that
> > does includes, now we do [for dependency tracking] - and sure we degrade
> > well.
> 
> The make implementation that does includes is only for developers of 
> the package.  It is not necessary to have a fancy make to build the 
> software.

It is if you want dependency tracking [and yes, one time builds
shouldn't need that, unless they ship with an unsettled graph]. As a
fraction, amongst your users, who do all of the following:
 - build their own binaries
 - do so with /no/ modifications to the code
 - on a platform with no suitable install program

Thats the key number - the amount of benefit that install-sh gives you.

> > All I'm suggesting is that the time has come to let folk on the small
> > proportion of machines without a sufficiently useful install, build it -
> > exactly as they have to build any other dependency they are lacking.
> 
> What other dependency might they be lacking?  My own package is quite 
> large but all of the dependencies are optional.

Lets start at the ridiculous and propose that they are missing a C
compiler.

-Rob


signature.asc
Description: This is a digitally signed message part

Re: how to detect broken install-sh?

[Robert Collins]

On Sun, 2009-09-27 at 16:00 -0500, Bob Friesenhahn wrote:
> On Sun, 27 Sep 2009, Robert Collins wrote:
> >
> > I suggest dropping install-sh completely except for the coreutils
> > package. coreutils is very portable, so its not unreasonable to require
> > that it is installed to locally build and install other packages.
> > coreutils of course cannot depend on itself being installed. A more
> 
> This seems like a pretty unreasonable requirement to me.  The 
> install-sh strategy has been working for quite a long time with hardly 
> any complaint until today.

The landscape has changed though, and I suspect that if we gather stats
about this we'll see that install-sh is dead weight for most packages
nearly all of the time.

Its true that it is not a lot of dead weight, but at some point we
should be raising the bar - ever so slightly - on what we bundle into
the tarball. At one point we never required a Make implementation that
does includes, now we do [for dependency tracking] - and sure we degrade
well.

All I'm suggesting is that the time has come to let folk on the small
proportion of machines without a sufficiently useful install, build it -
exactly as they have to build any other dependency they are lacking.

BTW, on solaris, /usr/ucb/install is apparently the right thing to use,
and has been there since SunOS 5.10  Last Revised 14 Sep 1992 :).

-Rob


signature.asc
Description: This is a digitally signed message part

Re: how to detect broken install-sh?

[Robert Collins]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ralf Wildenhues wrote:
 What would be the best way?  Do you think this might cause other
>>> problems?
>> I suggest dropping install-sh completely except for the coreutils
>> package.
> 
> Expecting GNU coreutils to be installed on each system is unreasonable.
> Other systems have quite well-functioning tools, too.  Autotools
> generally strive to produce packages that work well on all kinds of
> Posix and almost-Posix systems.
> 
> Cheers,
> Ralf

So I don't expect coreutils to be installed; I'm saying *packages other
than coreutils* should *depend on a working /usr/bin/install*.

Thats quite a different thing :)

- -Rob
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkq/SOwACgkQ42zgmrPGrq55bQCeKM3laGVIGbsvaxhH/pXpGvTl
AAwAoJt+Tye5QcVTZr8qRhVbxz4kan73
=4YWB
-END PGP SIGNATURE-

Re: how to detect broken install-sh?

[Robert Collins]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Brian Gough wrote:
> Hi,
> 
> I'd like to hear thoughts about the best way to detect a broken install-sh.
..
> Maybe it would be good to have a check for problems with install-sh.

I think that is a waste of cycles for every project except Automake :).

> I can see a couple of ways this could be done:
> 
>   - AC_PROG_INSTALL could confirm that the install program it finds
> works in the way it will be used in "make install" and give an
> error otherwise.

Perhaps it already does for the system install, if so extending that to
the bundled one isn't a great stretch.

> What would be the best way?  Do you think this might cause other
> problems?

I suggest dropping install-sh completely except for the coreutils
package. coreutils is very portable, so its not unreasonable to require
that it is installed to locally build and install other packages.
coreutils of course cannot depend on itself being installed. A more
conservative fix would be to keep install-sh for the transitive closure
of coreutils build dependencies (but given that one can cross compile I
think this is also unnecessary).

- -Rob
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkq/JCsACgkQ42zgmrPGrq6DcwCggZlqS67rlS71viXlFm8iM5pO
WMAAnj9sN8RhFgPRSEeancSsWDvH6pRv
=JJoX
-END PGP SIGNATURE-

Re: place automake files separately from source files

[Robert Collins]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


> This way people can build using the GNU automake system if they so desires
> and I do not overwrite the original non-automake Makefiles.   Then how can I
> specify the sources files in source1,c, etc.  Keep in mind that the original
> source tree may be complex with deep subdirectories of its own.

You can do this using relative paths to the sources in your Makefile.am
and configure.ac.

However I suggest you either adopt automake, or don't. Keeping 2 build
systems means you will have twice as many places to record changes to
build dependencies. Why do you want folk to be able to use the old build
system?

- -Rob
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkq+xzoACgkQ42zgmrPGrq7+3gCeOkztJP8Ucgl7P4u1M8YrgBjd
c7gAoLkZv6xUFYfh7Bvr6dPEJAB0BwtU
=u83k
-END PGP SIGNATURE-

--gnits, AC_INIT and VERSION

[Robert Collins]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

There was discussion about getting version numbers from VCS recently;
I've done a slightly different thing for a while now:
AC_DEFUN([SUBUNIT_MAJOR_VERSION], [0])
AC_DEFUN([SUBUNIT_MINOR_VERSION], [0])
AC_DEFUN([SUBUNIT_MICRO_VERSION], [2])
AC_DEFUN([SUBUNIT_VERSION],
[SUBUNIT_MAJOR_VERSION.SUBUNIT_MINOR_VERSION.SUBUNIT_MICRO_VERSION])
AC_PREREQ([2.59])
AC_INIT([subunit], [SUBUNIT_VERSION], [contact...])
AC_SUBST([SUBUNIT_MAJOR_VERSION]) ...

This works well, until gnits is added to AM_INIT_AUTOMAKE.

It looks like it expects a literal, not a macro in AC_INIT ?

It seems to me that for gnits to work with dynamic (whether from VCS or
otherwise) version numbers, it would need to actually get the expanded
form, or perhaps more sensibly provide a configure time check that the
version calculation *results* in a good version.

Something like:

all: am-version-check
.PHONY: am-version-check

am-version-check::


- -Rob

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkq51lsACgkQ42zgmrPGrq5moACcCOKo5fleZ2L7/q5IA3ChWLko
fcMAnjYikQhHp1AVTkA6UD0XjXnvjKV4
=lrJs
-END PGP SIGNATURE-

Re: distcheck and uninstall

[Robert Collins]

On Sat, 2009-09-19 at 08:33 +0200, Ralf Wildenhues wrote:

> No, I don't, but automake/NEWS indicates that it should've been around
> 1.7, and 'git show Release-1-7:lib/am/distdir.am' looks good, too.

Thanks again,
Rob


signature.asc
Description: This is a digitally signed message part

Re: distcheck and uninstall

[Robert Collins]

On Sat, 2009-09-19 at 08:24 +0200, Ralf Wildenhues wrote:
> Hello Robert,
> 
> * Robert Collins wrote on Sat, Sep 19, 2009 at 06:16:25AM CEST:
> > It would be nice if there was an option to tell automake not to (do
> > 'uninstall' as part of distcheck | require that uninstall leaves no
> > files behind)
> 
> You can do the latter by overriding either distuninstallcheck rule of
> the distuninstallcheck_listfiles command, see the lower half of
> <http://www.gnu.org/software/automake/manual/html_node/Checking-the-Distribution.html>.

Sweet, thanks. Do you recall when that target was added? [So I can make
sure that that version of automake is my required minimum..]

-Rob


signature.asc
Description: This is a digitally signed message part

distcheck and uninstall

[Robert Collins]

It would be nice if there was an option to tell automake not to (do
'uninstall' as part of distcheck | require that uninstall leaves no
files behind)

distcheck is very useful, it catches many distribution related bugs like
missing EXTRA_DIST and so on.

However, uninstall as a target is much less valuable - most users use
external packaging systems like rpm/deb. Further to that it can be time
consuming when integrating with non automake build systems such as
MakeMaker (via a Makefile.PL and -local & -hook rules such as those
appended) to get uninstall working at all - when they neither support
uninstall nor trivially expose appropriate hooks to implement it for
them.

As it stands, making distcheck work in cross-language packages where
external build tool integration is required, is a real nuisance because
of the requirement for uninstall to both work & not leave anything
behind.

There are two projects I'm involved in that this is causing some
nuisance; one is squid, where our uninstall target deliberately leaves
behind the users configuration file - uninstall vs purge, in debian
packaging concepts. The second is subunit, where a contributor has
provided some perl bindings and tools, but MakeMaker doesn't generate an
uninstall target - and even if it did files would likely be [correctly]
left behind, because a perl config file is edited as part of install.

[Please don't let the specific examples become a bikeshed :)]

-Rob
===
all-local: perl/Makefile
$(MAKE) -C perl all

check-local: perl/Makefile
$(MAKE) -C perl check

clean-local:
rm -f perl/Makefile

# Remove perl dir for VPATH builds.
distclean-local:
-rmdir perl > /dev/null
-rm perl/Makefile.PL > /dev/null

install-exec-local: perl/Makefile
$(MAKE) -C perl install

mostlyclean-local:
rm -rf perl/blib
rm -rf perl/pm_to_blib

uninstall-local:
-# there is no uninstall target! $(MAKE) -C perl uninstall

perl/Makefile: perl/Makefile.PL
mkdir -p perl
cd perl && perl Makefile.PL
-rm perl/Makefile.old > /dev/null



signature.asc
Description: This is a digitally signed message part

Re: invoke pkg-config with --static

[Robert Collins]

On Sat, 2009-05-23 at 18:18 +0200, Lorenzo Bettini wrote:
> Ralf Wildenhues wrote:


> > Of course, as soon as you propose your software for packaging at
> > debian.org, they will count not using .Private as bug ... ;-)
> 
> uh!  Good to know that!  Thanks :-)

This is because when you link against something that isn't directly
used, a library reference is made. This causes difficulty on upgrades to
the library-that-isn't-directly-used; and it can be very annoying to
maintainers of distributions.

-Rob



signature.asc
Description: This is a digitally signed message part

Re: My project can't use `silent-rules'

[Robert Collins]

On Sun, 2009-05-17 at 15:43 -0500, Bob Friesenhahn wrote:

> The reason why my package can not use AC_INIT is that the package 
> version information is (often) computed by shell script code based on 
> the last entry in the project ChangeLog or other information.  It is 
> (apparently) not possible for user-provided script code to be executed 
> prior to AC_INIT and it is not clear if AC_INIT would allow passing a 
> shell variable in order to obtain the package version value.

This isn't necessarily an answer, but its a step towards one. I've
recently started using the following consistently in my projects - I got
tired of repeating version numbers as hard coded strings in multiple
places.

AC_DEFUN([FOO_MAJOR_VERSION], [0])
AC_DEFUN([FOO_MINOR_VERSION], [0])
AC_DEFUN([FOO_MICRO_VERSION], [1])
AC_DEFUN([FOO_VERSION],
[FOO_MAJOR_VERSION.FOO_MINOR_VERSION.FOO_MICRO_VERSION])
AC_PREREQ([2.59])
AC_INIT([libfoo], [FOO_VERSION], [liboo-de...@example.com])
AC_CONFIG_SRCDIR([foo/foo.h.in])
AM_INIT_AUTOMAKE([-Wall -Werror foreign subdir-objects])
AC_CONFIG_MACRO_DIR([m4])
[FOO_MAJOR_VERSION]=FOO_MAJOR_VERSION
[FOO_MINOR_VERSION]=FOO_MINOR_VERSION
[FOO_MICRO_VERSION]=FOO_MICRO_VERSION
[FOO_VERSION]=FOO_VERSION
AC_SUBST([FOO_MAJOR_VERSION])
AC_SUBST([FOO_MINOR_VERSION])
AC_SUBST([FOO_MICRO_VERSION])
AC_SUBST([FOO_VERSION])

Its pretty verbose, when I get some more time to fiddle with build
systems rather than writing code I'm going to look at reducing the
redundancy - it should be one line for each FOO_VARIABLE.

Anyhow, the key thing is that defining FOO_VARIABLE as a function lets
you do pretty much what you want.

-Rob

-- 


signature.asc
Description: This is a digitally signed message part

Re: invoke pkg-config with --static

[Robert Collins]

On Sat, 2009-05-16 at 19:04 -0500, Bob Friesenhahn wrote:
> On Sat, 16 May 2009, Lorenzo Bettini wrote:
> 
> > when ./configure is run with --disable-shared, is there a way to invoke the 
> > pkg-config macro with --static (so that it does not select private 
> > libraries 
> > in the .pc file)?
> 
> It seems that LT_INIT has been executed, the shell environment 
> variables enable_shared and enable_static are set to 'yes' if 
> shared and/or static libraries will be built.  You can test these 
> environment variables to determine the parameter to pass to 
> pkg-config.

But note, that private libraries are likely needed to like against when
creating a static build; its only shared libraries that the runtime
linker can resolve missing dependencies transitively for you, [and that
only on some platforms like glibc linux].

-Rob


signature.asc
Description: This is a digitally signed message part

Re: rebuilding following a change in prefix?

[Robert Collins]

On Fri, 2009-05-08 at 06:52 +0200, Jan Engelhardt wrote:


> Well, automake (unfortunately?) does not currently issue a recompile
> when the compiler command changed.
> It would be really cool to have that, though.

Write the compiler command to a file (stamp-compiler). make things
depend on that file ;).

When determining compiler, do the temp, if_not_changed_leave_alone
dance.

-Rob



signature.asc
Description: This is a digitally signed message part

Re: Modifying CFLAGS for 'make distcheck'

[Robert Collins]

On Sat, 2008-02-09 at 14:52 -0600, Bob Friesenhahn wrote:
> On Sat, 9 Feb 2008, Ralf Wildenhues wrote:
> >
> > If *that* were still a concern for a compression tool (as opposed to
> > various vendor `tar' programs), then heck it should not be promoted at
> > all for wider use.  No, I don't think each package using Automake should
> > be turned into a regression test suite for lzma.  That's more fear,
> 
> Most of the compressors offer an option to test the compressed file. 
> For lzma, gunzip, and bunzip2, that is provided by the -t option, and 
> for zip, it is the -T option.  It is probably not a bad idea to at 
> least verify that the same tool thinks that the package can be 
> properly decompressed.

might be simpler to layer this:

dist-tar to get a tar
dist-$compressor builds a compressed version from that tar.

You can test each compressor has compressed correctly by decompressing
to a temp file and checking cmp $temptar $origtar

-Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Automatically Handling Tools that Produce Many Outputs

[Robert Collins]

On Mon, 2007-12-10 at 21:51 +, Olly Betts wrote:
> 
> I can't think of a way to easily dig out statistically useful data
> from a VCS or Google code search on how often it happens either to me
> or
> generally.  But I mainly offered it as a more concrete example of the
> sort of issues I had in mind.

One particularly common multiple-output case is generating the
dependencies used during compilation:

gcc -MD -o foo.c
-> foo.o
-> foo.d

:)

-Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Strictness

[Robert Collins]

On Sun, 2007-08-12 at 23:40 +0100, Noah Slater wrote:
> > I disagree. In a centralised VCS sure, you can scale to 100's of commits
> > a day - but in a distributed VCS - e.g. bzr, git, hg, monotone ... you
> > tend to get 100's of commits on branches, and a much smaller number of
> > branch merges occuring - branch merges being the point at which code
> > review is done, regression tests run etc. And its entirely appropriate
> > to create a human meaningful summary of the aggregate work done on that
> > branch.
> 
> Actually - this argument doesn't support your conclusion.
> 
> In Subversion (as an example) if you were merging branch A to the
> trunk after 10 commits you would have one revision entry for the merge
> - not the aggregate of all 10. So in this instance the commit message
> would be the description of the merge and the ChangeLog would (like I
> previously stated) make sense copying this exactly...

Well, in other VCS's the merged revisions are preserved. So 'log' shows
you the merge revision, and the log for the other revisions. Theres a
certain amount of repetition if you pull everything across, so what I've
seen a number of projects do is to do a fairly trivial commit message
and have the ChangeLog updated by the merge.

> ... which once again begs the question - why bother?

I think this boils down to 'do you think there is any disconnect between
commits to the VCS and telling humans whats happened from release to
release'.

I think there is.
-Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Strictness

[Robert Collins]

On Sat, 2007-08-11 at 22:06 +0200, Carl Fürstenberg wrote:
> On 8/11/07, Noah Slater <[EMAIL PROTECTED]> wrote:
> > > I think you misunderstanding me, it's the generation if the changelog
> > > that will take too long time.
> >
> > Well, yes - what else could I have understood from:
> >
> > > That not an optimal option, as it's illogical to store those files in the 
> > > svn.
> >
> > How long does it take to generate?
> >
> 
> it all depends on subjects connection and computer power, but for a
> normal person, perhaps 10 min.
> 
> I believe the ChangeLog file is a relic from the past when the
> developers made one commit per week generally, it doesn't really fit
> when there can be hundreds of commits per day.

I disagree. In a centralised VCS sure, you can scale to 100's of commits
a day - but in a distributed VCS - e.g. bzr, git, hg, monotone ... you
tend to get 100's of commits on branches, and a much smaller number of
branch merges occuring - branch merges being the point at which code
review is done, regression tests run etc. And its entirely appropriate
to create a human meaningful summary of the aggregate work done on that
branch. 

-Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Automake violations of the gnu coding conventions

[Robert Collins]

On Mon, 2007-06-18 at 17:27 -0700, K. Richard Pixley wrote:
> 
> My question today is... is there any hope of bringing automake
> generated 
> Makefiles back into line with the GNU coding standards so that these 
> applications will work once again? 

Use AM_MAINTAINER_MODE in your package; this will disable the rules to
rebuild Makefile.in etc unless --enable-maintainer-mode is supplied by
the user.

-Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Configuring automake says autoconf 2.58 or higher needed. Have au toconf 2.59 installed. What is/goes wrong?

[Robert Collins]

On Mon, 2005-01-17 at 03:18 +, Andrew Suffield wrote:
> Only the
> sender can do anything better than this, because they're the only one
> with the necessary information.

Its not at all clear to me that they have sufficient information.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Configuring automake says autoconf 2.58 or higher needed. Have au toconf 2.59 installed. What is/goes wrong?

[Robert Collins]

On Sun, 2005-01-16 at 07:01 -0500, Thomas Dickey wrote:
> On Sun, 16 Jan 2005, Ralf Corsepius wrote:
> 
> > On Sat, 2005-01-15 at 13:15 +0100, Alexandre Duret-Lutz wrote:
> >
> >> PS: I know this is not the first time, but I simply do not
> >> understand why you respond to bug reports without Cc: the
> >> reporter.
> > I normally respond CC:-ing the reporter on auto*.gnu.org lists, because
> > they tend to be unreliable. Not have done so in this case was just an
> > oversight.
> 
> otoh, when I do that, I usually get 2-3 complaints from people stating 
> that I shouldn't (ymmv).

They should configure their mail system appropriately so that they don't
see duplicates. (for example, see 'man formail'). MS Exchange, for
instance, does this automatically.

From rom what I recall of the mailing list headers, its not possible to tell
a priori who in the CC list is subscribed and who isn't, thus to ensure
that all interested parties do recieve a copy, I always reply-to-all.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: How to setup an example programs subdirectory?

[Robert Collins]

On Sat, 2005-01-01 at 20:24 -0500, Simon Perreault wrote:
> Hi,
> 
> I have a question for which I haven't been able to find an answer on my own, 
> using the usual resources (manual, google, etc).
> 
> My project uses automake and I want to have a directory containing example 
> programs. These programs should not be built on a simple "make", but could be 
> built on a "make examples" directive. How can I handle that?
> 
> Thanks!

IIRC - untested:
===
EXTRA_PROGRAMS=foo bar

examples: $(EXTRA_PROGRAMS)

===

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Automake and new tar

[Robert Collins]

On Thu, 2004-11-25 at 21:59 +0100, Christian Fredrik Kalager Schaller
wrote:
> Hi Automake hackers,
> 
> I am maintainer of a GNOME  module called gnome-themes-extras containing
> a set of metathemes for the GNOME desktop. After upgrading my distro I
> have been unable to 'make dist' gnome-themes-extras on Fedora due to tar
> complaining:
> tar: gnome-themes-extras-0.8.0/Gorilla/icons/scalable/mimetypes/gnome-
> mime-application-vnd.sun.xml.impress.svg: file name is too long (max
> 99); not dumped
> 
> I was told today that the problem is that the new tar actually follows
> the spec for the method that automake wants to use. So automake needs to
> be fixed to not use the -o option to tar. 
> 
> Is this a known problem, is there some bugzilla somewhere I can
> report this? Anyone know if/when a release which handles this is planed?

Apologies if I missed someone answering this, but its hitting us
(squid-cache) as well,  we can no longer do a make dist on up-to-date
systems.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: non-recursive make and tests

[Robert Collins]

On Mon, 2004-08-30 at 20:30 -0500, Bob Friesenhahn wrote:
> On Mon, 30 Aug 2004, Bob Friesenhahn wrote:
> >
> > It would be quite helpful if Automake offered a mode in which it 
> > automatically changed the working directory to the directory where the test 
> > program/script resides and set $srcdir to the relative position in the source 
> > tree to support VPATH builds.  This would emulate the operation of recursive 
> > builds.  Either a global Automake option could be used to enable this 
> > (subdir-tests), or a "RTESTS" (relative tests) mode would be provided.
> 
> It seems that this topic has not caught anyone's interest since there 
> have been no follow-up posts today.  Surely someone else has converted 
> their recursive project to a non-recursive project and noticed that it 
> is very difficult to get test suites working again?

I found it trivial. Mind you, all my tests are - by design - self
contained.

Rob


signature.asc
Description: This is a digitally signed message part

Re: RFC: doc for `Handling Tools that Produce Many Outputs'

[Robert Collins]

On Thu, 2004-02-05 at 10:36, Eric Siegerman wrote:

> I believe this fails on the following corner case.  Suppose the
> date ordering is like this (with data.h being the oldest):
>   data.h   data.foo   data.c
> 
> data.h is out of date with respect to data.foo, so one wants to
> rebuild it, but I don't think that will happen:


Then data.c is not derived from data.foo. Or someone has manually edited
it - either of which is incorrect for this scenario. Try touching
Makefile.am, then editing Makefile.in...

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: RFC: doc for `Handling Tools that Produce Many Outputs'

[Robert Collins]

On Sun, 2004-02-01 at 09:28, Alexandre Duret-Lutz wrote:
> This is a new section I'd like to add to the FAQ.  It has been
> discussed two or three times on the list.
> 
> I'm posting it here for comment.  (In fact I'm mainly hoping
> that some kind fellow will point out English mistakes...)

Cute. It reads fine to me.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: pathnames containing spaces

[Robert Collins]

On Thu, 2004-01-29 at 00:08, Earnie Boyd wrote:
> Good luck with fixing the white space problems in every process that 
> reads arguments and uses white space as a delimiter of some sort.

Earnie has a very good point - GNU Arch faces the same problem with a
limited set of tools - patch, diff and tar. (Plus it's own internals of
course). We've come to the conclusion that whilst fixing
spaces-in-filenames, it would be sensible to address unicode glyphs at
the same time - so as to prevent auditing code twice (once to escape
spaces, once to espace unicode).

It's a big project no matter how you look at it... 

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Expressing dependencies

[Robert Collins]

On Mon, 2004-01-05 at 03:53, Laurence Finston wrote:

> This is essentially what I tried to do by using the auxiliary program
> `3DLDFcpl' in the rule for building the executable `3dldf' (roughly):
> 
> 3dldf: $(3DLDF_CWEBS)
>3DLDFcpl

Thats not quite what I was suggesting.

> Not changing the timestamp of the .cxx file isn't enough. In fact, I change
> the name of the ctangle output from .c to .cxx based on
> whether the file has changed since the last version, and use the .cxx file in
> the rule for building the object files. This preserves the timestamp of the
> .cxx file, if the .c file doesn't contain any significant changes.
> 
> I actually had a problem with Automake assuming that 3DLDF was a C program
> rather than a C++ program because of the .c extension. Comparing the .c files
> to the .cxx files, renaming them, if appropriate, and using the .cxx files in
> the rules for building the .o files solves this problem. I think there's at
> least one other way of solving it, perhaps by means of an Autoconf variable,
> but I don't remember off-hand. 

Right, thats orthogonal though: if we 3dldf.o is built from 3dldf.cxx
and 3dldf.h, and 3dldf.cxx and 3dldf.h are built from wdlfd.web; then we
can focus on the dependency issue - not the actual extensions.

> It's reasonable behavior for Automake to assume that the sources for a C++
> program are called .cc, .cxx, or .c++, but it is
> very restrictive to assume files need to be rebuilt based merely on the
> information that their prerequisites have a more recent timestamp. 

As I said, thats the /only/ information (cheaply) available to automake.
Alternatives include generating a md5 of the file and comparing that to
a calculated one, or other such has-content-changed tests. But make
isn't language aware - it can't tell if a file change is 'meaningful'.  

> As Andrew
> Suffield pointed out in his posting, this problem affects Bison and Flex, too,
> which are probably used much more often for GNU software than CWEB.  I suspect
> there are other tools affected by this problem as well.

Yes - and the same problem applies - make assumes that the commands it
runs are for a single purpose - with no side effects.

> I don't think the problem lies with CWEB, or Bison and Flex, for that matter. 
> CWEB isn't GNU software, and I doubt whether the authors would appreciate
> being asked to "fix" it. We can't do anything about make, either. I think the
> problem should be solved within Automake. 

The problem lies in the concept of the header 'maybe changing'. If the
header is dependent on the .web source, then it can't be considered
correct if it's timestamp is older than the .web source - because make
is meant to look at just modification dates. If you want to consider
file content changes, make has to generate some database (of sorts) to
track file content, and while we *could* do that in GNU Make, automake
targets posix make - so that would not be portable. So the rules that
will most likely work well for you are - no target to build the .h file
at all, and magic in a script to replace it if needed. Note that if the
header is replaced, you'll need to reinvoke make on the same dir, to get
it to notice that reliably.

Rob


-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Expressing dependencies

[Robert Collins]

On Sun, 2004-01-04 at 08:17, Laurence Finston wrote:

> The problem is that make makes certain assumptions that don't apply when CWEB
> is used.

I think thats an incorrect statement. It would be more accurate to say
that CWEB hasn't been built with any thought to the impact on make. Make
has only the file system data available to it to determine 'has X
changed more recently than Y.' config.status for example, when it
regenerates config.h will only alter the file if the contents have
changed - so that it preserves the timestamp. I think that most
pre-preocessors in this sense could benefit from a wrapper of some sort
that would equally not alter the file IF nothing had changed - and you
could use that wrapper directly in make rules for ctangle.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: non recursive includes proof of concept #2

[Robert Collins]

On Sat, 2003-12-20 at 00:41, Alexandre Duret-Lutz wrote:
> >>> "Robert" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
> [...]
> 
>  Robert> It transforms macros and paths in an included file (called
>  Robert> Makefile.rules for now) , to make them suitable for a non-recursive
>  Robert> build.
> 
> I'm skeptical about whether this approach can be made to work
> intuitively.  More precisely I don't think it is generally
> possible to let users write a subdir Makefile.am fragment as if
> it would be run locally in the subdirectory, and translate
> *anything* so it actually works from another directory.  Let's
> mention user-defined rules referring rewritten variables, or
> flag variables including things such as -I.

Right. To have both in-dir and top-level Makefiles work may not be
possible. My key goal is the top level Makefile assembled from the
included fragments though - if the dual approach won't work.. it won't.
That said, if the user can do it with only minor hoops to jump
through...

> Anyway you asked for comments on the patch so here are some.
> (I'm sorry I had to be brief in the end, because I have a train
> to catch in one hour.  I'll be away for one week.)
> 
> I don't see what in your changes require the file to be named
> Makefile.rules.

It isn't required to be named that now - my original proof of concept 2
odd years ago started out with a fixed name, which I removed even before
posting back then. So - no requirement.

> This sounds neat.  But AFAICT no test case really cd into that
> given dir and perform the build.

I'll add a test case.

> [...]

> Judging from the comments, I understand that 
> bin_PROGRAMS = foo does not become path/foo?
> Or is it a typo in the example of @canonised_macro_lists?

ah. It's 
bin_PROGRAMS = foo
foo_SOURCES = bar.c
->
bin_PROGRAMS = path/foo
path_foo_SOURCES = path/bar.c

> Since these variables will be used for member ship check, better
> use a hash

Ok, will try this out.


> Is this required?  `include' automatically distributes it's
> argument, I presume you've preserved this for `subdir_include'.

Right, so it won't be required.

> Please use GNU-like 2-space indentation for new code (see HACKING).
> 
> The tail of all Perl files already setups the indentation style
> for Emacs.  Maybe you can submit similar hints for your editor.

Hmm, I thought I had.

I'll stop replying here - I'm going to implement your suggestions as I
get time.

(an aside: some of the issues are holdovers from the 2 year old patch
I'd resurrected).

I'll drop a note here when I've done your suggestions.

Cheers
Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.

Re: HEAD test suite - 6 failures.

[Robert Collins]

On Sat, 2003-12-20 at 00:47, Alexandre Duret-Lutz wrote:

>  Robert> that the scripts in lib/ aren't chmodded correctly.
> 
> Why aren't they?  How did they loose their permissions?

Errm, that was my fault. An oversight in a cvs extracting tool, that I
wasn't aware of at the time.

>  Robert> Perhaps 'make' should chmod them to the recommended mode ?
> 
> I'd prefer we find (and possibly fix) the guilty tool that
> resets permissions.

CVS doesn't manage permissions - it only duplicates then at file
creation time. It seems fragile to me to expect this permission ignorant
program to Do The Right Thing.

Still, my problem is fixed, and it won't bite me again.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: HEAD test suite - 6 failures.

[Robert Collins]

On Thu, 2003-12-18 at 20:00, Alexandre Duret-Lutz wrote:
> >>> "Robert" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
>  Robert> Are the following tests known to fail (on debian unstable):
> 
> Nein, no tests are known to fail.  What does VERBOSE=x say?

that the scripts in lib/ aren't chmodded correctly.

Perhaps 'make' should chmod them to the recommended mode ?

Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.


signature.asc
Description: This is a digitally signed message part

HEAD test suite - 6 failures.

[Robert Collins]

Are the following tests known to fail (on debian unstable):


FAIL: ccnoco.test
FAIL: gnits2.test
FAIL: gnits3.test
FAIL: pr300-lib.test
FAIL: pr300-prog.test
FAIL: python3.test

Cheers,
Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

non-recursive via includes

[Robert Collins]

Ok, 
I plan to push this through a little closer to completion (some
feedback from the maintainers would be greatly appreciated !)

I've created a branch for this in arch:

[EMAIL PROTECTED]/automake--nonrecursive--1.8

The arch repository is at http://people.initd.org/robertc/automake/

(GNU Arch is an RCS system - http://www.gnu.org/software/gnu-arch/)

I've also imported the CVS HEAD branch's entire history, to the branch
[EMAIL PROTECTED]/automake--HEAD--0. I'll be keeping HEAD
up to date, to keep my branch's delta minised. If there's interest I'm
happy to generate a branch for CVS-1.8 and keep that up to date.

Now, the patch status is where it was before - it passes the two test
cases I created. I'll be adding a couple more complex test cases and
seeing what breaks tonight.

If anyone is interested in commit messages for this repository, drop me
a line.

Cheers,
Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive make & maintenance issue

[Robert Collins]

On Wed, 2003-12-10 at 05:06, Tom Tromey wrote:

> It isn't impossible.  I once wrote up some ideas along these lines:
> 
> http://sources.redhat.com/ml/automake/2001-07/msg00248.html
> 
> Obviously I never got around to implementing this :-)

Have you looked at either of my proof-of-concepts?

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: failure building HEAD

[Robert Collins]

On Tue, 2003-12-02 at 21:44, Alexandre Duret-Lutz wrote:
> I think this is the problem.  Ben, you cannot write
> `$output.tmp' because when $output is /dev/null a user cannot
> create /dev/null.tmp.  This change breaks the configuration of
> all versions of Automake since 1.6 :(

Yah, so, the right way to do this would be to write to a safe temp file
in /tmp, and then move that to the destination?

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive make & maintenance issue

[Robert Collins]

On Tue, 2003-12-02 at 02:10, Bob Friesenhahn wrote:
> > Hmm, I'd prefer to do it via the include mechanism - see my crude, but
> > effective updated proof of concept - posted here a minute ago.
> 
> I like your include approach.  It helps convert existing recursive
> builds into non-recursive builds with minimum pain.  However, there
> are sufficient reasons to write only one Makefile (e.g. source tree is
> treated as "read only", or personal preference) that both mechanisms
> should be supported.

I'm not arguing against the single-big-file-method... but I am curious:
how does a 'read only' source tree affect this? If there is a
Makefile.am in it that you want to use without alteration, you can just
SUBDIRS= x y z   #not subdir
DIST_SUBDIRS=x y z subdir
subdir_include subdir/Makefile.am

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive make & maintenance issue

[Robert Collins]

On Tue, 2003-12-02 at 07:08, Bob Friesenhahn wrote:
> By 'read only', I mean that there is an existing source tree with no
> Makefile.am's (perhaps it uses some other build system) and you are
> not allowed to (or shouldn't) update it.  Since Automake supports
> subdirectories, the Makefile.am doesn't need to reside in the source
> tree and it doesn't care if files which would normally conflict with
> Automake already exist in the tree.
> 
> That was the case for my latest Automake expedition.

Ah, that makes more sense. If layered carefully the two approaches could
even be compatible.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

oops, regression - updated proof of concept - #2a

[Robert Collins]

A minor oversight led to a regression, which I caught when the test
cases finished running... here's a replacement patch. (Still use the
test cases from my previous email).

Rob
-- 
GPG key available at: .
Index: automake.in
===
RCS file: /cvs/automake/automake/automake.in,v
retrieving revision 1.1523
diff -u -p -r1.1523 automake.in
--- automake.in	30 Nov 2003 17:00:36 -	1.1523
+++ automake.in	1 Dec 2003 11:02:38 -
@@ -181,11 +181,27 @@ my $ELSE_PATTERN =
 my $ENDIF_PATTERN =
   '^endif(?:\s+(!?)\s*([A-Za-z][A-Za-z0-9_]*))?\s*(?:#.*)?' . "\$";
 my $PATH_PATTERN = '(\w|[/.-])+';
+my $INCLUDE_KEYWORD = 'include';
+my $SUBDIR_INCLUDE_KEYWORD = 'subdir_include';
 # This will pass through anything not of the prescribed form.
-my $INCLUDE_PATTERN = ('^include\s+'
+my $INCLUDE_PATTERN = ('^' . $INCLUDE_KEYWORD . '\s+'
 		   . '((\$\(top_srcdir\)/' . $PATH_PATTERN . ')'
 		   . '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
 		   . '|([^/\$]' . $PATH_PATTERN . '))\s*(#.*)?' . "\$");
+my $SUBDIR_INCLUDE_PATTERN = ('^' . $SUBDIR_INCLUDE_KEYWORD . '\s+'
+		   . '((\$\(top_srcdir\)/' . $PATH_PATTERN . ')'
+		   . '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
+		   . '|([^/\$]' . $PATH_PATTERN . '))\s*(#.*)?' . "\$");
+
+# Canonised variable suffixes
+my @canonised_macro_names =
+qw(SOURCES);
+# Canonised variable contents (foo->path/foo)
+my @canonised_macro_values = 
+qw(SOURCES);
+# Canonised macro lists (foo ->path_foo)
+my @canonised_macro_lists = 
+qw(PROGRAMS);
 
 # Match `-d' as a command-line argument in a string.
 my $DASH_D_PATTERN = "(^|\\s)-d(\\s|\$)";
@@ -216,7 +232,7 @@ my @common_files =
 	ansi2knr.1 ansi2knr.c compile config.guess config.rpath config.sub
 	configure configure.ac configure.in depcomp elisp-comp
 	install-sh libversion.in mdate-sh missing mkinstalldirs
-	py-compile texinfo.tex ylwrap),
+	py-compile texinfo.tex ylwrap Makefile.rules),
  @libtool_files, @libtool_sometimes);
 
 # Commonly used files we auto-include, but only sometimes.
@@ -1697,6 +1713,38 @@ sub handle_single_transform_list (@)
 return @result;
 }
 
+# $VALUE
+# transform_file_list ($PREPEND, @FILES)
+# 
+# insert $PREPEND before every file path that is not absolute
+#
+sub transform_file_list ($$)
+{
+  my ($prepend, $tmpfiles) = @_;
+  my $result = "";
+  my @files = ();
+  @files = split(/ /, $tmpfiles); 
+  while (scalar @files > 0)
+  {
+	$_ = shift @files;
+
+	if ($_ =~ s/^\$\(top_srcdir\)\///)
+	  {
+	$result .= " \$\(top_srcdir\)\/" . $_;
+	  }
+	  elsif ( $_ =~ s/^\$\(srcdir\)\///)
+	  {
+	$result .= " \$\(srcdir\)\/$prepend" . $_;
+	  }
+	  else
+	  {
+	$result .= " $prepend" . $_;
+	  }
+  }
+  verb "transformed value: '$result'\n";
+  return $result . "\n";
+}
+
 
 # $LINKER
 # define_objects_from_sources ($VAR, $OBJVAR, $NODEFINE, $ONE_FILE,
@@ -2145,7 +2193,7 @@ sub handle_programs
   # Canonicalize names and check for misspellings.
   my $xname = &check_canonical_spelling ($one_file, '_LDADD', '_LDFLAGS',
 	 '_SOURCES', '_OBJECTS',
-	 '_DEPENDENCIES');
+	 '_DEPENDENCIES', '_CFLAGS');
 
   $where->push_context ("while processing program `$one_file'");
   $where->set (INTERNAL->get);
@@ -2250,7 +2298,7 @@ sub handle_libraries
   # Canonicalize names and check for misspellings.
   my $xlib = &check_canonical_spelling ($onelib, '_LIBADD', '_SOURCES',
 	'_OBJECTS', '_DEPENDENCIES',
-	'_AR');
+	'_AR', '_CFLAGS');
 
   if (! var ($xlib . '_AR'))
 	{
@@ -2371,7 +2419,20 @@ sub handle_ltlibraries
   # Canonicalize names and check for misspellings.
   my $xlib = &check_canonical_spelling ($onelib, '_LIBADD', '_LDFLAGS',
 	'_SOURCES', '_OBJECTS',
-	'_DEPENDENCIES');
+	'_DEPENDENCIES', '_CFLAGS');
+
+# Tell the source code what library we are building
+#	my $tempvariable = '';
+#	if ( &variable_defined ($xlib . '_CFLAGS'))
+#	{
+#	# Define the lib_CFLAGS variable.
+#	$tempvariable .= &variable_value ($xlib . '_CFLAGS');
+#	&variable_delete ($xlib . '_CFLAGS');
+#	}
+#	my $libname_short = $xlib;
+#	$libname_short =~ s/_la$//  ;
+#	$libname_short = uc ($libname_short);
+#	&define_variable ($xlib . '_CFLAGS', ' -D' . $libname_short . '_COMPILATION ' . $tempvariable);
 
   # Check that the library fits the standard naming convention.
   my $libname_rx = "^lib.*\.la";
@@ -5413,19 +5474,62 @@ sub check_trailing_slash ($\$)
   return $$line =~ /\\$/;
 }
 
+# include ()
+# worker routine to include a file.
+#
+sub include(@)
+{
+  my ($path, $relative_dir, $where, $canonise, @include_stack) = @_;
+  my $prepend_path = "";
+
+  if ($path =~ s/^\$\(top_srcdir\)\///)
+{
+  push (@include_stack, "\$\(top_srcdir\)/$path");
+  error ("attempt to translate a top_sr

Re: Non-recursive make & maintenance issue

[Robert Collins]

On Fri, 2003-11-28 at 03:49, Jirka Hanika wrote:


> My view is that these (and other) problems disappear if you use a
> per-directory Makefile.am; but I also see the benefits (esp. compilation
> speed) of a non-recursive Makefile.  So the solution could be to support
> generating a single Makefile from multiple Makefile.am's in
> subdirectories.  (Just kidding.  But interested in seeing the reasons
> why this is nearly impossible.)

It's completely possible - 2 years ago I did a proof of concept:

http://mail.gnu.org/archive/html/automake/2001-08/txt7.txt

And I've refreshed that today.

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive make & maintenance issue

[Robert Collins]

On Fri, 2003-11-28 at 04:29, Bob Friesenhahn wrote:

> It is not a problem as long as Automake provides sufficient
> automatic translation capabilities.  There just needs to be a standard
> way to create definitions and refer to existing definitions, including
> those that Automake generates for its use.
> 
> In order to avoid confusion, Automake could adopt a GNU-make function
> style syntax which indicates where translations are required.
> 
> For example:
> 
>   $(xlate foo/bar++)_SOURCES
> 
> would be automatically translated by Automake to
> 
>   foo_bar___SOURCES
> 
> and substitution of existing Makefile defines (but not defines based
> on autoconf substituted values) should also work so the following
> should produce the same result:
> 
>   FOO_BAR=foo/bar++
>   $(xlate $(FOO_BAR))_SOURCES
> 
> This would be extremely useful since it would allow a package's
> directory organization to be re-arranged without excruciating pain.

Hmm, I'd prefer to do it via the include mechanism - see my crude, but
effective updated proof of concept - posted here a minute ago.

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: failure building HEAD

[Robert Collins]

On Mon, 2003-12-01 at 18:09, Alexandre Duret-Lutz wrote:
> >>> "Robert" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
> [...]
> 
>  Robert> configure:1847: cd conftest && eval autoconf -o /dev/null conftest.ac
>  Robert> autom4te: cannot open /dev/null.tmp: Permission denied
> 
>  Robert> Is this a 'need to use 2.59' thing? (If so, the error is misleading).
> 
> It works fine with Autoconf 2.58.   I wonder if your version of Autoconf
> is patched, or if autoconf is a wrapper, or something.  I can't see where
> this `.tmp' suffix would come from.

I'm on debian unstable, so there is a wrapper - but I bypassed it with
the same results.

Two things combined got it going for me:
1) ./bootstrap
2) doing an in-source-tree configure.

I'm not sure which one fixed it.

Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.

non recursive includes proof of concept #2

[Robert Collins]

Well, I finally snuck in a little time to update my proof of concept for
non recursive includes.

Still, I don't code perl - and it shows ;).

How to use?

Grab CVS automake, apply thepatch, drop the test files into tests
subdir.

Have a look at the test cases to see how to use it.

What does it do?

It transforms macros and paths in an included file (called
Makefile.rules for now) , to make them suitable for a non-recursive
build.

As show by the test cases, this allows a couple of neat things:
1) A stub Makefile.am 
===
include \$(srcdir)/Makefile.rules
===
is all that is needed in a given subdirectory to generate a full
makefile. (Useful if you want to be able to cd to a given dir and
perform builds just in that dir).

2) File paths, and canonical macro names are conveniently short - just
what Bob F has been (rightfully IMO) complaining about.

3) You don't end up with a huge Makefile.am to support, rather each part
of the project has a small rules file.

Rob

-- 
GPG key available at: .
Index: automake.in
===
RCS file: /cvs/automake/automake/automake.in,v
retrieving revision 1.1523
diff -u -p -r1.1523 automake.in
--- automake.in	30 Nov 2003 17:00:36 -	1.1523
+++ automake.in	1 Dec 2003 08:41:24 -
@@ -181,11 +181,27 @@ my $ELSE_PATTERN =
 my $ENDIF_PATTERN =
   '^endif(?:\s+(!?)\s*([A-Za-z][A-Za-z0-9_]*))?\s*(?:#.*)?' . "\$";
 my $PATH_PATTERN = '(\w|[/.-])+';
+my $INCLUDE_KEYWORD = 'include';
+my $SUBDIR_INCLUDE_KEYWORD = 'subdir_include';
 # This will pass through anything not of the prescribed form.
-my $INCLUDE_PATTERN = ('^include\s+'
+my $INCLUDE_PATTERN = ('^' . $INCLUDE_KEYWORD . '\s+'
 		   . '((\$\(top_srcdir\)/' . $PATH_PATTERN . ')'
 		   . '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
 		   . '|([^/\$]' . $PATH_PATTERN . '))\s*(#.*)?' . "\$");
+my $SUBDIR_INCLUDE_PATTERN = ('^' . $SUBDIR_INCLUDE_KEYWORD . '\s+'
+		   . '((\$\(top_srcdir\)/' . $PATH_PATTERN . ')'
+		   . '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
+		   . '|([^/\$]' . $PATH_PATTERN . '))\s*(#.*)?' . "\$");
+
+# Canonised variable suffixes
+my @canonised_macro_names =
+qw(SOURCES);
+# Canonised variable contents (foo->path/foo)
+my @canonised_macro_values = 
+qw(SOURCES);
+# Canonised macro lists (foo ->path_foo)
+my @canonised_macro_lists = 
+qw(PROGRAMS);
 
 # Match `-d' as a command-line argument in a string.
 my $DASH_D_PATTERN = "(^|\\s)-d(\\s|\$)";
@@ -216,7 +232,7 @@ my @common_files =
 	ansi2knr.1 ansi2knr.c compile config.guess config.rpath config.sub
 	configure configure.ac configure.in depcomp elisp-comp
 	install-sh libversion.in mdate-sh missing mkinstalldirs
-	py-compile texinfo.tex ylwrap),
+	py-compile texinfo.tex ylwrap Makefile.rules),
  @libtool_files, @libtool_sometimes);
 
 # Commonly used files we auto-include, but only sometimes.
@@ -1697,6 +1713,38 @@ sub handle_single_transform_list (@)
 return @result;
 }
 
+# $VALUE
+# transform_file_list ($PREPEND, @FILES)
+# 
+# insert $PREPEND before every file path that is not absolute
+#
+sub transform_file_list ($$)
+{
+  my ($prepend, $tmpfiles) = @_;
+  my $result = "";
+  my @files = ();
+  @files = split(/ /, $tmpfiles); 
+  while (scalar @files > 0)
+  {
+	$_ = shift @files;
+
+	if ($_ =~ s/^\$\(top_srcdir\)\///)
+	  {
+	$result .= " \$\(top_srcdir\)\/" . $_;
+	  }
+	  elsif ( $_ =~ s/^\$\(srcdir\)\///)
+	  {
+	$result .= " \$\(srcdir\)\/$prepend" . $_;
+	  }
+	  else
+	  {
+	$result .= " $prepend" . $_;
+	  }
+  }
+  verb "transformed value: '$result'\n";
+  return $result . "\n";
+}
+
 
 # $LINKER
 # define_objects_from_sources ($VAR, $OBJVAR, $NODEFINE, $ONE_FILE,
@@ -2145,7 +2193,7 @@ sub handle_programs
   # Canonicalize names and check for misspellings.
   my $xname = &check_canonical_spelling ($one_file, '_LDADD', '_LDFLAGS',
 	 '_SOURCES', '_OBJECTS',
-	 '_DEPENDENCIES');
+	 '_DEPENDENCIES', '_CFLAGS');
 
   $where->push_context ("while processing program `$one_file'");
   $where->set (INTERNAL->get);
@@ -2250,7 +2298,7 @@ sub handle_libraries
   # Canonicalize names and check for misspellings.
   my $xlib = &check_canonical_spelling ($onelib, '_LIBADD', '_SOURCES',
 	'_OBJECTS', '_DEPENDENCIES',
-	'_AR');
+	'_AR', '_CFLAGS');
 
   if (! var ($xlib . '_AR'))
 	{
@@ -2371,7 +2419,20 @@ sub handle_ltlibraries
   # Canonicalize names and check for misspellings.
   my $xlib = &check_canonical_spelling ($onelib, '_LIBADD', '_LDFLAGS',
 	'_SOURCES', '_OBJECTS',
-	'_DEPENDENCIES');
+	'_DEPENDENCIES', '_CFLAGS');
+
+# Tell the source code what library we are building
+#	my $tempvariable = '';
+#	if ( &variable_defined ($xlib . '_CFLAGS'))
+#	{
+#	# Define the lib_CFLAGS variable.
+#	$tempvariable .= &variable

failure building HEAD

[Robert Collins]

checking whether autoconf is installed... yes
checking whether autoconf works... no
configure: error: The installed version of autoconf does not work.
Please check config.log for error messages before this one.

I get the above configuring CVS automake.

from config.log:

configure:1819: eval autoconf --version
autoconf (GNU Autoconf) 2.58
Written by David J. MacKenzie and Akim Demaille.

Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is
NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
configure:1822: $? = 0 
configure:1830: result: yes
configure:1840: checking whether autoconf works
configure:1847: cd conftest && eval autoconf -o /dev/null conftest.ac
autom4te: cannot open /dev/null.tmp: Permission denied

Is this a 'need to use 2.59' thing? (If so, the error is misleading).

autoconf, invoked directly on other projects, works -just fine-.

Rob

-- 
GPG key available at: .

Re: Non-recursive make & intermediate objects

[Robert Collins]

On Sat, 2003-11-22 at 07:12, Bob Friesenhahn wrote:

> So this bug is only present if SUBDIRS is used to cause the Makefile
> to also have a recursive aspect.

Yes - which projects that include other projects will need. Or for
things like test scripts, I find throwing them in a sandbox of sorts
much easier than a full recursive makefile - at least until I get back
to that proof of concept.

Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive make & intermediate objects

[Robert Collins]

On Thu, 2003-11-20 at 09:50, Bob Friesenhahn wrote:
> On Thu, 20 Nov 2003, Robert Collins wrote:
> 
> > subdir_objects in your automake options.
> >
> > Problem is, there is a design headache that makes recursive clean fail
> > with this approach - I forget the bug #, but it's on my todo, waay down
> > there :p.
> 
> Ahhh, subdir-objects.  Since this is so important to non-recursive
> makes, it would be useful if it was referenced in the "An Alternative
> Approach to Subdirectories" section of the documentation.
> 
> Please move the clean bug up in the priority level.  Automake has a
> non-recursive user now. :-)

We've had them for ages - I've been using it for 2 years now in
progressive increasing sizes I keep meaning to get back to my
transforming include patch
(http://sources.redhat.com/ml/automake/2001-08/msg00112.html) to make
authoring them less unpleasant.

PR 373 is the bug I was referring to on clean - it's a general race
condition.
http://mail.gnu.org/archive/html/automake/2003-07/msg00064.html is a
relevant email in this list archives.

> I suspect/believe that libtool will have some problems as well.

libtool has worked fine for me, with non recursive make for 2 years now.
Not to say there are not issues to find :}.

Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.


signature.asc
Description: This is a digitally signed message part

Re: Non-recursive make & intermediate objects

[Robert Collins]

On Thu, 2003-11-20 at 09:04, Bob Friesenhahn wrote:
> Using Automake 1.7.9, I am attempting to create a single Makefile.am
> which is capable of building all of the libraries used by the project.
> The source files to the project are located in subdirectories, and the
> output libraries should also be located in subdirectories.  The
> objective is to replace an existing recursive build.
> 
> Using a rule like:
> 
> noinst_LIBRARIES = libs/somedir/libfoo.a
> libs_somedir_libfoo_a_SOURCES=foo.cc
> 
> and then inpecting the output of 'make -n' (and the generated
> Makefile), I see evidence that the build will put all .Po files in a
> .deps subdirectory under the Makefile.am, and all the .o files in the
> same directory as Makefile.am.  This approach seems quite wrong to me
> since it is quite possible that libraries and applications may use
> similarly named source files.  Intermediate files should be placed
> either under the directory where the library is placed, or in the
> directory where the source files live.  An even better solution would
> allow the user to specify where intermediate files are placed on a
> per-library and per application basis.

subdir_objects in your automake options.

Problem is, there is a design headache that makes recursive clean fail
with this approach - I forget the bug #, but it's on my todo, waay down
there :p.

Rob

Re: Aborting automake?

[Robert Collins]

On Sat, 2003-11-08 at 11:22, Harlan Stenn wrote:
> I have a situation where I want every Makefile.am to 'include' one of
> several files.
> 
> If none of these files are 'include'd I want the automake run to abort.
> 
> I know how to cause the abort at runtime, but I'd rather catch this problem
> while automake is running.

If you want the automake include facility, then you'd need to extend
automake to allow some sort of policy check at the end of processing
each Makefile.am. thinking out loud, something like
check_script=${top_srcdir)/link_Makfile.am.sh in automake_options...
then have automake call out to that script, with a failure meaning abort
the automake run.

Rob

-- 
GPG key available at: .

Re: precompiled header suggestion

[Robert Collins]

On Wed, 2003-10-01 at 04:30, Tom Tromey wrote:
> Recently gcc added precompiled header support.  This is mostly useful
> for C++, but C might benefit in some cases too.

Waay cool.

Are you planning on doing this, or just sketching the design and hoping
for volunteer contributions?

What might be a useful starting point is some manual test cases or
sample rules, to aim for.

Rob

-- 
GPG key available at: .

Re: Should -i mkdir?

[Robert Collins]

On Sat, 2003-09-27 at 02:20, Alexandre Duret-Lutz wrote:

>  adl> autopoint and libtoolize usually run before automake
>  adl> and put things into this directory too.  So if some tools has to
>  adl> create the directory, I think it should be autopoint.
> 
> Sorry, I meant "it should be autoreconf".

/if it's used/


Rob

-- 
GPG key available at: .

Re: convenience binaries

[Robert Collins]

On Mon, 2003-09-22 at 22:31, Andrew Suffield wrote:
> On Mon, Sep 22, 2003 at 10:01:24PM +1000, Robert Collins wrote:
> > On Mon, 2003-09-22 at 21:22, Warren Turkal wrote:
> > > Robert Collins wrote:
> > > > yes,
> > > > noinst_PROGRAMS = convenience_binaries
> > > 
> > > Can these convenience programs be built for the host arch in a
> > > cross-compiled environment?
> > 
> > probably, you'll likely need to override the default build recipe
> > though.. I haven't tried, perhaps someone else here has more details.
> 
> Can it ever be correct for a noinst object to be built for the target
> environment? By definition, they should only exist on the build
> system.

Not necessarily - but certainly the common case is for them to be build
system only.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.

Re: convenience binaries

[Robert Collins]

On Mon, 2003-09-22 at 21:22, Warren Turkal wrote:
> Robert Collins wrote:
> > yes,
> > noinst_PROGRAMS = convenience_binaries
> 
> Can these convenience programs be built for the host arch in a
> cross-compiled environment?

probably, you'll likely need to override the default build recipe
though.. I haven't tried, perhaps someone else here has more details.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.

Re: convenience binaries

[Robert Collins]

On Mon, 2003-09-22 at 19:56, Warren Turkal wrote:
> Is there any support in automake for building a binary that will only be
> used during the build process?

yes,
noinst_PROGRAMS = convenience_binaries

any rules that depend on one of the binaries should be written as:

thing: binary$(EXEEXT)
./binary 

Rob

-- 
GPG key available at: .

race condition with subdir objects:

[Robert Collins]

the following will break on distclean aith automake 1.7.5:

Makefile.am:

SUBDIRS=a
AUTOMAKE_OPTIONS = subdir-objects

bin_PROGRAMS=foo
foo_SOURCES=a/foo.cc

a/Makefile.am
bin_PROGRAMS=bar
bar_SOURCES=bar.cc


The failure is because subdirs are distcleaned first, and a/.deps is rm
-rf'd before the top level makefile runs.

Possible solutions include:
distclean prefix first, not postfix.
change the distclean process to not rm -rf the .deps dir.

I don't have time to hack up a test case just yet... sorry.

Cheers,
Rob

Re: about vpath problems

[Robert Collins]

On Thu, 2002-08-15 at 02:01, Tom Tromey wrote:
> > ">" == leiming xd <[EMAIL PROTECTED]> writes:
> 
> >> In win32 platforms ,the path of one file can include blank
> >> characters,I want to know how to add this path in the vpath.
> 
> I imagine it may not be possible.  If it can work, autoconf needs to
> be taught how.  Could you run experiments by quoting the path name in
> various ways and report back to us?

I've used this quite a bit on unix and win32 (cygwin build and host)
without any errors that I recall. I haven't explicitly tried recently
though.

Cheers,
Rob

Re: top-level Makefile required?

[Robert Collins]

On Sat, 2002-08-03 at 10:02, Harlan Stenn wrote:
> Bug, I'd guess.
> 
> Why does automake/autoconf assume it is "in charge" of the directory
> structure?

I'll leave this to the core guys to answer. My understanding is that
thats what automake is designed to handle though..

Rob

Re: Pathalogical behavior with "more" AM_CONDITIONAL()s?

[Robert Collins]

On Sat, 2002-08-03 at 09:50, Harlan Stenn wrote:
> I've done a bit more testing.
> 
> The slowdown happens if I only modify 1 Makefile.am, and it seems to be
> related to using SUBDIRS inside an AM_CONDITIONAL.
> 
> If I change the Makefile.am to use a non-SUBDIRS variable inside the
> conditional autmake zips right along.

Can you post your example?

The usual approach to SUBDIRS is something like:

DIST_SUBDIRS = every subdir you may or may not want
SUBDIRS = every compulsory subdir @optionalsubdirlist1@
@optionalsubdirlist2@


Are you doing something different?

Rob

Re: top-level Makefile required?

[Robert Collins]

On Sat, 2002-08-03 at 09:55, Harlan Stenn wrote:
> Is it a bug or a feature that when using automake+autoconf there Must be a
> top-level Makefile?
> 
> I tried to write a test case for automake (debugging the AM_CONDITIONAL
> slowdown problem I'm seeing) and I wrote a top-level configure.ac that only
> specified the following outputs:
> 
>  AC_CONFIG_FILE(a/Makefile)
>  AC_CONFIG_FILE(b/Makefile)
>  AC_OUTPUT
> 
> and when I ran automake on this I never got either of those Makefile.in's
> generated.
> 
> What gives?

Feature I guess :}. It doesn't make a lot of sense to have a
non-existence top level Makefile, as you'd never recurse down the tree
:}.

Rob

Re: Pathalogical behavior with "more" AM_CONDITIONAL()s?

[Robert Collins]

On Thu, 2002-08-01 at 12:55, Harlan Stenn wrote:
> Here are the results of my testing:
> 
>  someconditionals/:
>   automake-1.5:249.480u 2.660s 4:42.35 89.3%  0+0k 0+0io 341pf+0w
>   automake-1.6.3:  341.810u 2.840s 6:07.24 93.8%  0+0k 0+0io 356pf+0w
> 
>  moreconditionals/:
>   automake-1.5:   1134.130u 6.730s 21:27.70 88.5% 0+0k 0+0io 615pf+0w
>   automake-1.6.3: 1199.000u 5.520s 21:45.32 92.2% 0+0k 0+0io 578pf+0w
> 
> "someconditionals" uses 24 conditionals and has 632 Makefile.am's.
> 
> "moreconditionals" uses 24 conditionals and has 635 Makefile.am's.
> 
> It sure looks like I ran these on the same machine, and I doubt the
> difference is due to 3 more Makefile.am's...

I should have provided more information: Automake 1.6 will run slower,
but produce -much- smaller Makefile.in files.

1.5 generates (2^conditionalcount)*rules with any conditional lines.
1.6 eliminates non-impacting permutations.

One example: an automake 1.5 generated Makefile.in from a 8K Makefile.am
for squid (www.squid-cache.org) was 1.6 Megabytes in size.
The same file generated via automake 1.6 is 40K in size.

So... automake is slower to run, but generates faster makefiles.

Also, you should only need to run automake infrequently over the whole
project.

Finally, to address your 800 second increase - you don't say whether the
someconditionals results are from a subset of the moreconditionals run -
assuming they are then it appears you only have 3 Makefile.am's to
investigate to identify the problem file - and I'm sure that it's likely
worth generating a problem report (PR) for if we can pin down the
Makefile layout that causes such a large increase.

I.e. what is the timing of:
project root$ automake (your options) path/to/one/slow/Makefile.am. 
Is that still ~1000 seconds? Or is it ~200->400 seconds?

Rob



signature.asc
Description: This is a digitally signed message part

Re: Pathalogical behavior with "more" AM_CONDITIONAL()s?

[Robert Collins]

On Thu, 2002-08-01 at 09:34, Harlan Stenn wrote:
> I have a report that indicates that as the number of AM_CONDITIONAL()s
> increases, the time it takes to run automake increases Significantly.
> 
> This is with automake-1.5.
> 
> I'm about to dive in and look at what's going on to be sure, but just in
> case, does anybody have any information on what might be going on here?

It's a bug in 1.5, use 1.6.

Rob





signature.asc
Description: This is a digitally signed message part

Re: Option no-gzip

[Robert Collins]

On Tue, 2002-07-30 at 19:02, Alexandre Duret-Lutz wrote:
> >>> "Robert" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
> [...]
> 
>  Robert> Still I don't see how that could be accomplished with Bruce's
>  Robert> suggestion of multiple targets.
> 
> IMHO ./configure is not a exactly the right interface to make
> distribution policy decisions.

Oh, I'm easy about *where*.

Rob

Re: Option no-gzip

[Robert Collins]

On Tue, 2002-07-30 at 18:11, Alexandre Duret-Lutz wrote:
> >>> "Robert" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
> [...]
> 
>  Robert> This means:
>  Robert> build dist tree
>  Robert> compress with compressor 1
>  Robert> compress with compressor 2
>  Robert> clean dist tree
> 
> This is the current behavior of `make dist'.

Cool. Still I don't see how that could be accomplished with Bruce's
suggestion of multiple targets.

Rob

Re: Option no-gzip

[Robert Collins]

On Mon, 2002-07-29 at 13:14, Bruce Korb wrote:
> Robert Collins wrote:
> > 
> > On Sat, 2002-07-27 at 02:25, Bruce Korb wrote:
> > > Akim Demaille wrote:
> > > >
> > > > Would that be accepted?  For some of my projects, I don't need nor
> > > > want the .gz, I just want the .bz2.
> > >
> > > If you are going to parameterize it at all, then parameterize it
> > > completely.  e.g.  --compressor=bzip2 [ --compression-suffix=bz2 ]
> > > except there would be a table of defaults for the suffix:
> > 
> > I like this idea. One tweak:
> > pluralise the command:
> > --compressors=bzip2,gzip[,...]
> 
> Meaning you would produce more than one tarball?
> Such an option would need something more explicit.
> e.g. "make bzip2-dist gzip-dist ..."

Why? I am suggesting that 
make dist
create one tarball for each compressor it has been told about. 

It seems to me that if you want individual targets, they *could* be
created, but that the most common option will be to want a set number of
tarballs, and to build them all at one time.

This means:
build dist tree
compress with compressor 1
compress with compressor 2
clean dist tree

as opposed to the current (or last I checked closely)
build dist tree
compress with gz
clean dist tree
build dist tree
compress with bz2
clean dist tree

which is less efficient than it could be.

Rob



msg05738/pgp0.pgp
Description: PGP signature

Re: Option no-gzip

[Robert Collins]

On Sat, 2002-07-27 at 02:25, Bruce Korb wrote:
> Akim Demaille wrote:
> > 
> > Would that be accepted?  For some of my projects, I don't need nor
> > want the .gz, I just want the .bz2.
> 
> If you are going to parameterize it at all, then parameterize it
> completely.  e.g.  --compressor=bzip2 [ --compression-suffix=bz2 ]
> except there would be a table of defaults for the suffix:

I like this idea. One tweak:
pluralise the command:
--compressors=bzip2,gzip[,...]

Rob

Re: Alternate automake output?

[Robert Collins]

===
- Original Message -
From: "William Robertson" <[EMAIL PROTECTED]>
To: "Tom Tromey" <[EMAIL PROTECTED]>

> I know this is a hack, but could automake play along with this, and
> would this work?  Alternatively, is there a cleaner way to achieve
this
> goal?

I'd just use subdir_objects and include the full paths to the module
code - much as per the style of 'recursive make considered harmful'.

Rob

RE: monolithic Makefile.am

[Robert Collins]

> -Original Message-
> From: Tom Tromey [mailto:[EMAIL PROTECTED]] 
> Sent: Sunday, May 12, 2002 9:15 AM
> To: Robert Collins
> Cc: Richard Boulton; Harlan Stenn; Automake
> Subject: Re: monolithic Makefile.am
> 
> 
> >>>>> "Rob" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
>
> I looked at this.  I think it is missing one of the major 
> features I'd like out from `import', namely the ability to 
> have targets like `subdir/clean' and `subdir/all'.

Are targets like 
foo/all:
portable?

Emitting 
foo_all:
would be trivial.

Emitting foo/all: may be more difficult, I really don't know.

I've update the patch to be against 1.6, and to emit
foo_bar_SOURCES: 
for the rule
bar_SOURCES = 
in a subdir Makefile.rules.
 
> I think doing this requires the bulk of the work, since it 
> means a lot of code generation changes in automake.  I'd be 
> delighted to be proven wrong :-)

--- ../automake-1.6/automake.in Wed Mar  6 15:46:14 2002
+++ automake.in Sun May 12 13:34:00 2002
@@ -157,6 +157,11 @@ my $INCLUDE_PATTERN = ('^include\s+'
   . '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
   . '|([^/\$]' . $PATH_PATTERN. '))\s*(#.*)?$');
 
+my $SUBDIR_INCLUDE_PATTERN = ('^subdir_include\s+'
+   . '((\$\(top_srcdir\)/' . $PATH_PATTERN . ')'
+  . '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
+  . '|([^/\$]' . $PATH_PATTERN. '))\s*(#.*)?$');
+
 # Some regular expressions.  One reason to put them here is that it
 # makes indentation work better in Emacs.
 my $AC_CONFIG_AUX_DIR_PATTERN = 'AC_CONFIG_AUX_DIR\(([^)]+)\)';
@@ -208,7 +213,7 @@ my @common_files =
ansi2knr.1 ansi2knr.c compile config.guess config.rpath
config.sub
configure configure.ac configure.in depcomp elisp-comp
install-sh libversion.in mdate-sh missing mkinstalldirs
-   py-compile texinfo.tex ylwrap),
+   py-compile texinfo.tex ylwrap Makefile.rules),
  @libtool_files, @libtool_sometimes);
 
 # Commonly used files we auto-include, but only sometimes.
@@ -2121,6 +2126,39 @@ sub handle_single_transform_list (@)
 return @result;
 }
 
+# $VALUE
+# transform_file_list ($PREPEND, @FILES)
+# 
+# insert $PREPEND before every file path that is not absolute
+#
+sub transform_file_list ($$)
+{
+  my ($prepend, $tmpfiles) = @_;
+  my $result = "";
+  my @files = ();
+  @files = split(/ /, $tmpfiles); 
+  while (scalar @files > 0)
+  {
+   $_ = shift @files;
+
+   if ($_ =~ s/^\$\(top_srcdir\)\///)
+ {
+   $result .= " \$\(top_srcdir\)\/" . $_;
+ }
+ elsif ( $_ =~ s/^\$\(srcdir\)\///)
+ {
+   $result .= " \$\(srcdir\)\/$prepend" . $_;
+ }
+ else
+ {
+   $result .= " $prepend" . $_;
+ }
+  }
+   print "result:$result\n";
+  return $result . "\n";
+}
+ 
+
 # $BOOL
 # define_objects_from_sources ($VAR, $OBJVAR, $NODEFINE, $ONE_FILE,
 #  $OBJ, $PARENT, $TOPPARENT)
@@ -2223,6 +2261,8 @@ sub define_objects_from_sources ($$$
 }
 
 
+
+
 # $OBJNAME
 # subobjname ($VARNAME)
 # -
@@ -2889,7 +2929,7 @@ sub handle_ltlibraries
# Canonicalize names and check for misspellings.
my $xlib = &check_canonical_spelling ($onelib, '_LIBADD',
'_LDFLAGS',
  '_SOURCES', '_OBJECTS',
- '_DEPENDENCIES');
+ '_DEPENDENCIES',
'_CFLAGS');
 
if (! variable_defined ($xlib . '_LDFLAGS'))
{
@@ -2897,6 +2937,19 @@ sub handle_ltlibraries
&define_variable ($xlib . '_LDFLAGS', '');
}
 
+   # Tell the source code what library we are building
+#  my $tempvariable = '';
+#  if ( &variable_defined ($xlib . '_CFLAGS'))
+#  {
+#  # Define the lib_CFLAGS variable.
+#  $tempvariable .= &variable_value ($xlib . '_CFLAGS');
+#  &variable_delete ($xlib . '_CFLAGS');
+#  }
+#  my $libname_short = $xlib;
+#  $libname_short =~ s/_la$//  ;
+#  $libname_short = uc ($libname_short);
+#  &define_variable ($xlib . '_CFLAGS', ' -D' . $libname_short .
'_COMPILATION ' . $tempvariable);
+
# Check that the library fits the standard naming convention.
my $libname_rx = "^lib.*\.la";
if ((variable_d

RE: per object cflags

[Robert Collins]

Done I think.. the GNATS web returned to the entry screen without giving
me a PR number, so I'm not sure..

Rob

> -Original Message-
> From: Tom Tromey [mailto:[EMAIL PROTECTED]] 
> Sent: Monday, May 06, 2002 3:38 PM
> To: Robert Collins
> Cc: [EMAIL PROTECTED]
> Subject: Re: per object cflags
> 
> 
> >>>>> "Rob" == Robert Collins <[EMAIL PROTECTED]> writes:
> 
> Rob> Are there any plans to allow per object CFLAGS (and CXXFLAGS...)?
> 
> You mean per-`.o'?  There aren't any concrete plans.  Offhand 
> I can't think of any barrier to doing it though.  In fact it 
> might be easy. (Explaining it might be hard.)
> 
> Could you submit a PR for this feature?
> 
> Rob> Currently I compile those source files with manual rules, but... 
> Rob> that means that the dependency tracking stuff doesn't work for 
> Rob> them...
> 
> Yeah, this is a problem.  It would be nice if we had a 
> relatively easy way to handle situations where you want the 
> full rule with some minor tweak.  Duplicating the code is 
> pretty bad, since it is complicated and changes from time to time.
> 
> Tom
> 

RE: lex & yacc with C++ projects

[Robert Collins]

> -Original Message-
> From: Tom Tromey [mailto:[EMAIL PROTECTED]] 
> Sent: Monday, May 06, 2002 3:45 PM

> Do you have other C files that you want to compile with 
> $(CC)?  If not you could set `CC = $(CXX)'.  (This is pretty ugly.)

We have a .c file that generates linker stubs for objects in .dll's that
may not be present on the users system. That also has specific CFLAGS.
Yep, twice in one project.
 
Rob

RE: monolithic Makefile.am

[Robert Collins]

In fact, here are some of the references...

http://sources.redhat.com/ml/automake/2001-08/msg00061.html
http://sources.redhat.com/ml/automake/2001-08/msg00088.html
http://sources.redhat.com/ml/automake/2001-08/msg00109.html
http://sources.redhat.com/ml/automake/2001-08/msg00113.html

Msg 113 highlights one of the nasties that does turn up - local user
ruels of the 
@ for foo in $(MACRO) form.

http://sources.redhat.com/ml/automake/2001-08/msg00112.html
is the proof-of-concept I sent in.

Have fun,
Rob

RE: monolithic Makefile.am

[Robert Collins]

> -Original Message-
> From: Tom Tromey [mailto:[EMAIL PROTECTED]] 
> Sent: Sunday, May 05, 2002 11:18 AM

> Long term I'd like us to ease this sort of thing.
> 
> My working idea is to have a new `import' command which is 
> like `include' but understands about directory structure.  So 
> for instance it would know that `bin_PROGRAMS = foo' in 
> subdir would really mean `bin_PROGRAMS = subdir/foo'.  Also 
> all the normal targets would have subdir versions, so you 
> could invoke `make subdir/mostlyclean'.

I hacked up an import command. It was simply a variant on include, that
appended the include prefix to any relative paths. It worked so-so, but
I didn't have time to make it work well enough to submit it. Anyway, for
someone who actually understands automakes innards, it should be pretty
simple: refactor the standard include to perform canonicalisation with a
handed-in prefix, and then for normal include statements, use no prefix,
for imports use the path to the importee as the prefix. I don't have the
code around now, but IIRC I sent a proof-of-concept patch to the list.
 
> There are a lot of details to solve.  And the implementation 
> will be hard, since it will touch basically everything.

It didn't need to when I fiddled. The things that I ran into have
largely been solved now (subdir_objects etc were more buggy then). All
the rules are subdir and other dir aware, so it's only the
transformation of the include that matters. And I accomplished that by
setting the current logical dir to the one the importee was in before
processing, and restoring the logical dir afterwards.

Rob

RE: Sending _OBJECTS output to configuration specific directory - HELP!!

[Robert Collins]

Run the configure script twice.

Once from $(srcdir)/build/Release with CFLAGS=-O3 CXXFLAGS=-O3
Once from $(srcdir)/build/Debug with CFLAGS="-O -g" CXXFLAGS="-O -g"

Cheers,
Rob

RE: lex & yacc with C++ projects

[Robert Collins]

> -Original Message-
> From: Guido Draheim [mailto:[EMAIL PROTECTED]] 
> Sent: Friday, April 26, 2002 9:49 PM

> hhh. even though I need some enlightment what's wrong with a
> libstdc++ dependency for a c++ compiled source - so your project
> uses c++ files without libstdc++ and you want to include a parser 
> but that would bring you an unwanted lib-dependency?

Yes.
 
> anyway, since I'm thinking about it, I am wondering if the 
> following could work - what do you think about such rules...
> 
> .lo.tab.c :
>$(LTCXXCOMPILE) -c $<
> 
> .o.tab.c :
>$(CXXCOMPILE) -c $<
> 
> ... or does that not work with `make`? hmmm, just wondering...

These are the current rules:

fooparse.cc fooparse.h: fooparse.y
bison -d -o fooparse.cc $(srcdir)/fooparse.y
@mv fooparse.cc.h fooparse.h 2>/dev/null || mv fooparse.hh
fooparse.h

foolex.cc: foolex.l fooparse.h
flex -8 $(srcdir)/foolex.l
mv lex.yy.c foolex.cc

BUILT_SOURCES = fooparse.cc foolex.cc

... and in the PROGRAMS fooparse.cc foolex.cc fooparse.h

what would be great would be

in the _PROGRAMS fooparse.y foolex.l

LEX_LANG = CXX
YACC_LANG = CXX

or something similar (with the explicit behaviour that the lexx and yacc
mode will still be std C output.)

Rob

RE: lex & yacc with C++ projects

[Robert Collins]

> -Original Message-
> From: Guido Draheim [mailto:[EMAIL PROTECTED]] 
> Sent: Friday, April 26, 2002 8:57 PM
> To: Robert Collins; [EMAIL PROTECTED]
> Subject: Re: lex & yacc with C++ projects
> 
> 
> Es schrieb Robert Collins:
> > 
> > It would be nice to be able to tell automake that we want 
> to compile 
> > the out of lex and yaxx with g++, not gcc. (this is for C++ 
> projects).
> 
> rename mysource.l -> mysource.ll
> rename mysource.y -> mysource.yy
> 
> and that's it. Were the manual pages confusing about that?
> Or is there another problem that is not obvious to me...

Yes, the C++ output from the tools is incorrect for what we are doing.
(It depends on libstdc++). So all we want is the C output, and then
compile with C++.

Rob

per object cflags

[Robert Collins]

Are there any plans to allow per object CFLAGS (and CXXFLAGS...)?

I've got a projec that I want to put -Werror into the AM_C[XX]FLAGS
variable, but a couple of files won't compile without warnings. The
warnings are harmless, almost compiler bugs in fact, so fixing the
source isn't feasible there.

Currently I compile those source files with manual rules, but... that
means that the dependency tracking stuff doesn't work for them...

Rob

RE: lex & yacc with C++ projects

[Robert Collins]

> -Original Message-
> From: Robert Collins 
> Sent: Friday, April 26, 2002 7:18 PM
> To: [EMAIL PROTECTED]
> Subject: lex & yacc with C++ projects
> 
> 
> It would be nice to be able to tell automake that we want to 
> compile the out of lex and yaxx with g++, not gcc. (this is 
> for C++ projects).
> 
> Is there any way to do this currently?
> 
> At the moment I've got the .cc files listed as BUILT_SOURCES, 
> and custom rules to make them. 

I had two issues, and got confused when writing the email. The
BUILT_SOURCES solution works fine.. so please ignore the next bit.

Rob

> This isn't very satisfactory, 
> as depcomp doesn't then operate (or I copy the exact rule 
> from Makefile.in and get tightly bound to a version of automake...)
> 
> Rob
> 
> 

lex & yacc with C++ projects

[Robert Collins]

It would be nice to be able to tell automake that we want to compile the
out of lex and yaxx with g++, not gcc. (this is for C++ projects).

Is there any way to do this currently?

At the moment I've got the .cc files listed as BUILT_SOURCES, and custom
rules to make them. This isn't very satisfactory, as depcomp doesn't
then operate (or I copy the exact rule from Makefile.in and get tightly
bound to a version of automake...)

Rob

RE: non-recursive project example

[Robert Collins]

> -Original Message-
> From: Alexandre Duret-Lutz [mailto:[EMAIL PROTECTED]] 
> Sent: Tuesday, April 23, 2002 10:48 PM
> To: Robert Collins
> Cc: Tim Waugh; [EMAIL PROTECTED]
> Subject: Re: non-recursive project example
> 
> 
> >>> "Robert" == Robert Collins 
> <[EMAIL PROTECTED]> writes:
> 
> [...]
> 
>  Robert> When did it get fixed? 
>  >> 
>  >> 1.6.1
> 
>  Robert> That'll be it then. I'll leave it as it is until 1.5 
>  Robert> fades from use... 
> 
> Maybe I was not clear: if you simply remove the `VERSION=' 
> prefix it will work as you intend (and as documented) with 
> *all* versions of Automake.

Cool. Ok, will do.

Rob

RE: non-recursive project example

[Robert Collins]

> -Original Message-
> From: Alexandre Duret-Lutz [mailto:[EMAIL PROTECTED]] 
> Sent: Tuesday, April 23, 2002 10:26 PM
> To: Robert Collins
> Cc: Tim Waugh; [EMAIL PROTECTED]
> Subject: Re: non-recursive project example
> 
> 
> >>> "Robert" == Robert Collins 
> <[EMAIL PROTECTED]> writes:
> 
>  Robert> When did it get fixed? 
> 
> 1.6.1

That'll be it then. I'll leave it as it is until 1.5 fades from use... 

Rob

RE: non-recursive project example

[Robert Collins]

> -Original Message-
> From: Alexandre Duret-Lutz [mailto:[EMAIL PROTECTED]] 
> Sent: Tuesday, April 23, 2002 10:07 PM
> To: Robert Collins
> Cc: Tim Waugh; [EMAIL PROTECTED]
> Subject: Re: non-recursive project example
> 
> 
> >>> "Robert" == Robert Collins 
> <[EMAIL PROTECTED]> writes:
> 
> [...]
> 
>  Robert> AUTOMAKE_OPTIONS = dist-bzip2 subdir-objects VERSION=1.5
> 
> Where does this `VERSION=1.5' usage comes from?  That the 
> second time I see it.  AFAICT this relies on a bug in 
> Automake which has been fixed in 1.6.1.  This should be

When did it get fixed? If it wasn't fixed in 1.5, then my usage comes
from getting it to work under 1.5.

Rob

RE: non-recursive project example

[Robert Collins]

Here's one..
I've got another more complete example with installable libraries and
headers if needed, but it's somewhat longer. This is a trimmed down file
from a current project.

Rob

## Process this file with automake to produce Makefile.in
#
# $Id: Makefile.am,v 1.3 2002/01/13 14:16:17 robertc Exp $
#

AUTOMAKE_OPTIONS = dist-bzip2 subdir-objects VERSION=1.5
DISTCLEANFILES = include/stamp-h include/stamp-h[0-9]*

bin_PROGRAMS = foo

noinst_LTLIBRARIES = src/bar.la 

INCLUDES = -I $(top_srcdir)/include

noinst_HEADERS = include/bar.h 

foo_SOURCES = src/foo.cc 
foo_LDADD = src/bar.la 

src_bar_la_SOURCES = src/bar.cc
src_bar_la_LDFLAGS = -module

RE: Creting extra executables.

[Robert Collins]

Always create the gui makefile. 

Use a configure substitution to change the value of SUBDIRS, and use
DIST_SUBDIRS to ensure that all the code gets distributed.

Rob

RE: FW: sed: 33: conftest.s1: unescaped newline inside substitutepattern

[Robert Collins]

> -Original Message-
> From: Alexandre Duret-Lutz [mailto:[EMAIL PROTECTED]] 
> Sent: Thursday, April 18, 2002 6:58 PM

> Yes, it's fixed in 1.6.

Thanks.

Rob

FW: sed: 33: conftest.s1: unescaped newline inside substitute pattern

[Robert Collins]

w/ Automake 1.5, we have the following bug report. In summary, the
following shell code:

  am_aux_dir=`CDPATH=:; cd $ac_aux_dir && pwd`

is not portable to MacOS X, and is causing a headache for folk building
in the same dir tree. Can we change it to

am_aux_dir=`unset CDPATH; cd $ac_aux_dir && pwd` ?

or has this been already been reported and fixed in 1.6?

Rob

> -Original Message-
> From: Henrik Nordstrom [mailto:[EMAIL PROTECTED]] 
> Sent: Thursday, April 18, 2002 6:12 PM
> To: Duane Wessels
> Cc: [EMAIL PROTECTED]
> Subject: Re: sed: 33: conftest.s1: unescaped newline inside 
> substitute pattern
> 
> 
> Duane Wessels wrote:
> 
> > Here's the line from configure that sets $am_aux_dir:
> > 
> > am_aux_dir=`CDPATH=:; cd $ac_aux_dir && pwd`
> > 
> > The problem appears to be that 'cd' is printing the long version of 
> > the directory because it found it in the CDPATH list (":" being 
> > interpreted as the current directory).
> > 
> > I found two ways to fix this:
> > 
> > am_aux_dir=`unset CDPATH; cd $ac_aux_dir && pwd`
> > 
> > am_aux_dir=`CDPATH=:; cd ./$ac_aux_dir && pwd`
> 
> 
> Ah, now I know why I cannot reproduce the problem. I always 
> build in an object tree separate from the source.
> 
> The second approach cannot be used. $ac_aux_dir is outside 
> the current directory when building in a separate object tree.
> 
> Robert: Can you verify with the Automake people if changing 
> from "CDPATH=:" to "unset CDPATH" is OK?
> 
> And perphaps this should be reported as a bug to Apple and to 
> the zsh maintainers.. the manpage clearly states that the 
> shell only echos the new path found via cdpath in interactive 
> shells, and the above is not an interactive shell invocation...
> 
>cd [ directory ]
> [...]
>   mat of CDPATH is the same as that of PATH. In an
>   interactive shell, the cd command will print out
>   the name of the directory that it actually switched
>   to if this is different from the name that the user
>   gave.  These may be different either because the
>   CDPATH mechanism was used or because a symbolic
>   link was crossed.
> 
> The bug is confirmed in zsh version 3.0.7 (RedHat 6.2), 3.0.8 
> (MAC OS/X), 4.0.2 (RedHat 7.2), but is only a problem for us 
> if /bin/sh is a zsh (OS/X)
> 
> Regards
> Henrik
> 

RE: nobase_ room for improvement

[Robert Collins]

> -Original Message-
> From: Alexandre Duret-Lutz [mailto:[EMAIL PROTECTED]] 
> Sent: Sunday, April 14, 2002 8:15 PM

> As a generalization of this and Tom's _NAME, what about:
> 
> nobase_include_HEADERS = ...
> nobase_include_HEADERS_TRANSFORM = s,^foo/bar/delta/,,
> 
> This would be supported for programs and librararies too.

Sounds good.

Rob

.la files from outside the current project

[Robert Collins]

When I put the following:

bin_PROGRAMS = foo

foo_SOURCES = src/foo.cc
foo_LDADD = libbar.la

into a Makefile.am, the foo_DEPENDENCIES target gets libbar.la added -
even though it's not included in the source. libbar.la is present in
/usr/lib or /usr/local/lib. How can I avoid the auto-setting of
foo_DEPENDENCIES ?

Rob

RE: nobase_ room for improvement

[Robert Collins]

> -Original Message-
> From: Tom Tromey [mailto:[EMAIL PROTECTED]] 
> Sent: Sunday, April 14, 2002 10:40 AM

> Rob> While defining a new target to be $(includedir)/foo lets 
> you work 
> Rob> around this, it would be great to do something like: 
> Rob> nobase_preserve_foo_HEADERS = ...
> 
> I have some questions about this.  What if `foo' appears 
> twice in the directory name?  What if `foo' doesn't appear?  
> Is this the most useful way to handle this feature or is 
> there another way?  In any case something along these lines 
> does make sense.  The question (for me anyway) is specifying 
> it properly.

Yeah. I fiddled around with various ideas before posting. A different
one was

nobase_include_HEADERS = ...
include_HEADERS_strip = foo/bar/delta
where foo/bar/delta is removed from the path.
 
> I've been thinking of adding a _NAME macro that would let you 
> do this.  For instance:
> 
> bin_PROGRAMS = my-install
> 
> my_install_SOURCES = ...
> my_install_NAME = install
> 
> In the build tree we would build `my-install', but at install 
> time we would run the _NAME through the program transform and 
> then install the program under the resulting name.

As folk do that anyway with --program-transform-name=, a more granular
mechanism should be fairly straightforward. I like it.

 > We would do this for programs.  It would be nice to do it for 
> libraries (it has come up more than once) but it means 
> problems if programs are linked against them in the build 
> tree.  Maybe we could implement it by building the library 
> under the real name and having the _NAME name be just a 
> stamp-like file.
> 
> Any comments on this?

Sounds reasonable. What about this:

For libraries, could you tell automake to use a different name for
whatever builtin is conflicting? i.e. (using install as an example (say
I am building a module called install)).

AM_builtin_install = realinstall

and from there on in, automake uses realinstall instead of install in
all it's targets - including the install (now realinstall) target. 

Rob

distcheck with libtool

[Robert Collins]

Using 
$ autoconf --version
autoconf (GNU Autoconf) 2.53
$ automake --version
automake (GNU automake) 1.6
$ libtoolize --version
libtoolize (GNU libtool) 1.4e

With the appended Makefile.am and configure.in, make distcheck fails
with
===
ERROR: files left after distclean:
./src/.libs/GetOption.o
./src/.libs/Option.o
./src/.libs/BoolOption.o
./src/GetOption.o
./src/Option.o
./src/BoolOption.o
./testoption
./a.exe
make[1]: *** [distcleancheck] Error 1


Is this something that libtool or automake is responsible for?

==Makefile.am==
## Process this file with automake to produce Makefile.in
#
# $Id: Makefile.am,v 1.3 2002/01/13 14:16:17 robertc Exp $
#

AUTOMAKE_OPTIONS = dist-bzip2 subdir-objects VERSION=1.5
##DIST_SUBDIRS  = src
##SUBDIRS   = src

DISTCLEANFILES = include/stamp-h include/stamp-h[0-9]*

AM_CXXFLAGS = -I$(top_srcdir)/include

getoptincludedir = $(includedir)/getopt++

lib_LTLIBRARIES = libgetopt++.la
check_PROGRAMS = testoption

libgetopt___la_SOURCES = src/GetOption.cc src/Option.cc
src/BoolOption.cc
libgetopt___la_LDFLAGS = -lstdc++ -version-info 0:0:0

getoptinclude_HEADERS = include/Option.h include/GetOption.h \
include/BoolOption.h

testoption_SOURCES = tests/testoption.cc
testoption_LDADD = libgetopt++.la

== configure.in ==
dnl
dnl  Configuration input file for GetOpt++
dnl
dnl  Robert Collins, [EMAIL PROTECTED]
dnl
dnl  $Id: configure.in,v 1.5 2002/03/01 12:14:39 robertc Exp $
dnl
dnl
dnl
AC_INIT(src/GetOption.cc)
AC_PREREQ(2.52)
AC_CONFIG_AUX_DIR(cfgaux)
AM_INIT_AUTOMAKE(barch, 0.0.1-DEVEL)
AM_CONFIG_HEADER(include/autoconf.h)
AC_REVISION($Revision: 1.5 $)dnl
AC_PREFIX_DEFAULT(/usr/local/barch)
AM_MAINTAINER_MODE

dnl Set default LDFLAGS
if test -z "$LDFLAGS"; then
LDFLAGS="-g"
fi

PRESET_CFLAGS="$CFLAGS"

dnl Check for GNU cc
AC_LANG_CPLUSPLUS
dnl AC_API_WIN32
AC_PROG_CXX
AM_PROG_CC_C_O
AC_CANONICAL_BUILD
AC_CANONICAL_HOST
AC_PROG_LIBTOOL

CRYPTLIB=''
REGEXLIB='' # -lregex
LIBREGEX='' # libregex.a

dnl find out the exe extension for this platform. If it's not empty, use
it for CGI's as well.
AC_EXEEXT
AC_OBJEXT

AC_CHECK_HEADERS( \
unistd.h \
string   \
string.h
)

AC_C_CONST
AC_C_BIGENDIAN

AC_TYPE_SIZE_T 
AC_CHECK_TYPE(ssize_t, int)
AC_TYPE_PID_T
AC_TYPE_MODE_T

dnl Check for special functions
AC_FUNC_ALLOCA

dnl Check for needed libraries

AC_CHECK_LIB(regex, main, [REGEXLIB="-lregex"])
AC_CHECK_LIB(m, main)

dnl Check for libcrypt
dnl We use crypt(3) which may be in libc, or in libcrypt (eg FreeBSD)
AC_CHECK_LIB(crypt, crypt, [CRYPTLIB="-lcrypt"])
AC_SUBST(CRYPTLIB)

dnl Check for pthreads
dnl We use pthreads when doing ASYNC I/O
AC_CHECK_LIB(pthread, main, [PTHREADLIB="-lpthread"])
AC_SUBST(PTHREADLIB)

dnl Check for librt
dnl We use AIO in the coss store
AC_CHECK_LIB(rt, aio_read, [AIOLIB="-lrt"])
AC_SUBST(AIOLIB)

dnl Check for library functions
dnl AC_CHECK_FUNCS(\
dnl )


AC_OUTPUT([\
Makefile
])

nobase_ room for improvement

[Robert Collins]

Just an idea.

I use automake to make non-recursive build environments fairly routinely
now. (Recursive-make-considered-bad ...)

Installation of headers in subdirectories is more complex than it could
be (IMO).

The nobase_ prefix allows utilising an existing directory tree, but a
non-recursive build will almost always have a directory or two below the
Makefile that aren't desired on install.

While defining a new target to be $(includedir)/foo lets you work around
this, it would be great to do something like:

nobase_preserve_foo_HEADERS = ...

and have it achieve the same result.

Anyway, just some food for thought.

Cheers,
Rob