Re: Built sources always regenerated

[Braden McDaniel]

On Sun, 2006-01-01 at 15:41 -0700, Tom Tromey wrote:
> > "Braden" == Braden McDaniel <[EMAIL PROTECTED]> writes:
> 
> Braden> Forget about BUILT_SOURCES and *_DEPENDENCIES. The sources I'm 
> building
> Braden> get #include'd by browser.cpp. As such, checking of browser.cpp's
> Braden> dependencies should cause them to get (re)generated, right?
> 
> Braden> But it doesn't. If I remove BUILT_SOURCES, the files don't get
> Braden> generated before browser.cpp gets compiled. Why not?
> 
> Automake dependency tracking information is computed as a side effect
> of compilation.  So, the first time things are compiled, it has no way
> of knowing about dependencies on generated files.

Yeah... I did some more thorough looking into how automake does its
thing and discovered this.

> BUILT_SOURCES is a hack to get around this.  It basically inserts a
> 'make $(BUILT_SOURCES)' before targets like 'all'.  (It would be nice
> to have per-{executable,library} BUILT_SOURCES...)
> 
> I didn't look into this too deeply but I would guess that it is more
> make-related than automake-related.  You can add rules to the
> Makefile.am to help with debugging, eg:
> 
> hack:
> $(MAKE) $(BUILT_SOURCES)
> 
> Then in theory 'make hack' should reproduce the problem you're seeing.

And it does.

> If that doesn't happen then something weird and perhaps
> automake-related is going on.  If it does happen, you can try plain
> old Makefile debugging with 'make -d'.



The problem is that antlr doesn't seem to modify the timestamp when it
regenerates files.

Solved by touching the generated files immediately after antlr runs.



Thanks for your help, gentlemen.

-- 
Braden McDaniel   e-mail: <[EMAIL PROTECTED]>
Jabber: <[EMAIL PROTECTED]>

Re: Built sources always regenerated

[Tom Tromey]

> "Braden" == Braden McDaniel <[EMAIL PROTECTED]> writes:

Braden> Forget about BUILT_SOURCES and *_DEPENDENCIES. The sources I'm building
Braden> get #include'd by browser.cpp. As such, checking of browser.cpp's
Braden> dependencies should cause them to get (re)generated, right?

Braden> But it doesn't. If I remove BUILT_SOURCES, the files don't get
Braden> generated before browser.cpp gets compiled. Why not?

Automake dependency tracking information is computed as a side effect
of compilation.  So, the first time things are compiled, it has no way
of knowing about dependencies on generated files.

BUILT_SOURCES is a hack to get around this.  It basically inserts a
'make $(BUILT_SOURCES)' before targets like 'all'.  (It would be nice
to have per-{executable,library} BUILT_SOURCES...)

I didn't look into this too deeply but I would guess that it is more
make-related than automake-related.  You can add rules to the
Makefile.am to help with debugging, eg:

hack:
$(MAKE) $(BUILT_SOURCES)

Then in theory 'make hack' should reproduce the problem you're seeing.
If that doesn't happen then something weird and perhaps
automake-related is going on.  If it does happen, you can try plain
old Makefile debugging with 'make -d'.

Tom

Re: Built sources always regenerated

[Braden McDaniel]

Alright, let's back up a bit.

Forget about BUILT_SOURCES and *_DEPENDENCIES. The sources I'm building
get #include'd by browser.cpp. As such, checking of browser.cpp's
dependencies should cause them to get (re)generated, right?

But it doesn't. If I remove BUILT_SOURCES, the files don't get generated
before browser.cpp gets compiled. Why not?

-- 
Braden McDaniel   e-mail: <[EMAIL PROTECTED]>
Jabber: <[EMAIL PROTECTED]>

Re: Built sources always regenerated

[Braden McDaniel]

On Sun, 2006-01-01 at 11:29 +0100, Stepan Kasal wrote:
> Hello Braden,
> 
> I tried to check out openvrml from sourceforge, but anonymous cvs access
> doesn't seem to work well at the moment.

:-/

> But I was able to see the Makefile.am, and the obvious question is:
> whe creates
>   openvrml/src/libopenvrml/openvrml/Vrml97TokenTypes.txt
> and when?

It's not generated; it's manually created and modified. From my working
copy:

$ stat src/libopenvrml/openvrml/Vrml97TokenTypes.txt
  File: `src/libopenvrml/openvrml/Vrml97TokenTypes.txt'
  Size: 1040Blocks: 16 IO Block: 4096   regular file
Device: fd01h/64769dInode: 19466443Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  500/  braden)   Gid: (  500/  braden)
Access: 2005-12-31 17:03:59.0 -0500
Modify: 2005-06-29 04:34:58.0 -0400
Change: 2005-07-15 04:00:52.0 -0400

> Perhaps you didn't meant to work in directory
>   openvrml/src/libopenvrml/openvrml/
> but in the top level openvrml?

No; I did intend to put things in src/libopenvrml/openvrml.

> Then you would need to use the following:
>   $(top_builddir)/openvrml/Vrml97Parser.hpp
>   $(top_builddir)/openvrml/Vrml97Parser.cpp
>   $(top_srcdir)/openvrml/Vrml97Parser.g
>   $(top_builddir)/openvrml/Vrml97TokenTypes.txt
>   etc.
> 
> Please read about top_builddir, srcdir, and such in the Autoconf manual
> (and also perhaps in the Automake manual).

I know about them. As I mentioned, I'm generating these built sources
into srcdir because I want them to be distributed in the tarball. It's
my understanding that putting them in srcdir is the proper thing in that
case.

-- 
Braden McDaniel   e-mail: <[EMAIL PROTECTED]>
Jabber: <[EMAIL PROTECTED]>

Re: Built sources always regenerated

[Stepan Kasal]

Hello Braden,

I tried to check out openvrml from sourceforge, but anonymous cvs access
doesn't seem to work well at the moment.

But I was able to see the Makefile.am, and the obvious question is:
whe creates
  openvrml/src/libopenvrml/openvrml/Vrml97TokenTypes.txt
and when?

Perhaps you didn't meant to work in directory
  openvrml/src/libopenvrml/openvrml/
but in the top level openvrml?

Then you would need to use the following:
  $(top_builddir)/openvrml/Vrml97Parser.hpp
  $(top_builddir)/openvrml/Vrml97Parser.cpp
  $(top_srcdir)/openvrml/Vrml97Parser.g
  $(top_builddir)/openvrml/Vrml97TokenTypes.txt
  etc.

Please read about top_builddir, srcdir, and such in the Autoconf manual
(and also perhaps in the Automake manual).

Hope this helps,
Stepan Kasal

Re: Built sources always regenerated

[Braden McDaniel]

On Sat, 2005-12-31 at 14:22 +0100, Stepan Kasal wrote:
> Hello,
> 
> On Sat, Dec 31, 2005 at 02:51:30AM -0500, Braden McDaniel wrote:
> > but the behavior is the same: the sources are regenerated every time,
> > regardless of whether the antlr grammar has been modified.
> 
> I can imagine several problems:
> 1) there is a prerequisite which is always refreshed and causes this

I can't see how that could be happening. For instance, here's one of the
rules that generates sources from a grammar:

$(srcdir)/openvrml/Vrml97Parser.hpp 
$(srcdir)/openvrml/Vrml97Parser.cpp: $(srcdir)/openvrml/Vrml97Parser.g 
$(srcdir)/openvrml/Vrml97TokenTypes.txt
$(ANTLR) -o $(srcdir)/openvrml $(srcdir)/openvrml/Vrml97Parser.g

None of the prerequisites are getting updated. In fact, if I issue the
command:

$ make -C src/libopenvrml 
../../../src/libopenvrml/openvrml/Vrml97Parser.cpp

I get:

make: Entering directory 
`/home/braden/src/openvrml/openvrml/build/src/libopenvrml'
make: `../../../src/libopenvrml/openvrml/Vrml97Parser.cpp' is up to 
date.
make: Leaving directory 
`/home/braden/src/openvrml/openvrml/build/src/libopenvrml'

And yet if I then just issue "make", the source gets regenerated.

> 2) If you use non-GNU make, it can may not understand that "./foo.c" and
> "foo.c" is the same file.

I am using GNU make 3.80.

> 3) If you work on a networked drive, the target may have time stamp in the
> future, and make decides to refresh it.

Definitely not the issue.

> I cannot guess more without seeing the makefile.  Could you please post
> (an URL to) a self-contained example?  If you are able to cut some parts
> and make the example smaller, that would definitely help.

The problem Makefile.am is here:




Information for checking out the project from CVS is here:



The Makefile.am isn't terribly complex; but I imagine I can come up with
a simple example project if necessary.

> > I have some sources that are generated with antlr. I have custom make
> > rules that generate the sources from the antlr grammar. I have tried
> > listing the generated sources in both BUILT_SOURCES and _DEPENDENCIES,
> 
> Overriding *_DEPENDENCIES is dangerous, it is better to let Automake
> compute it.

Yes; when trying that, I checked to see what was "normally" in
*_DEPENDENCIES and I augmented it.

> The BUILT_SOURCES hack is useful sometimes, but it's better to avoid it.

Using BUILT_SOURCES works marginally worse than *_DEPENDENCIES: with
BUILT_SOURCES, the generated sources are regenerated *twice*:

make[3]: Entering directory 
`/home/braden/src/openvrml/openvrml/build/src/libopenvrml'
/usr/bin/antlr -o ../../../src/libopenvrml/openvrml 
../../../src/libopenvrml/openvrml/Vrml97Parser.g
ANTLR Parser Generator   Version 2.7.4   1989-2004 jGuru.com
/usr/bin/antlr -o ../../../src/libopenvrml/openvrml -glib 
../../../src/libopenvrml/openvrml/Vrml97Parser.g 
../../../src/libopenvrml/openvrml/X3DVrmlParser.g
ANTLR Parser Generator   Version 2.7.4   1989-2004 jGuru.com
make  all-am
make[4]: Entering directory 
`/home/braden/src/openvrml/openvrml/build/src/libopenvrml'
/usr/bin/antlr -o ../../../src/libopenvrml/openvrml 
../../../src/libopenvrml/openvrml/Vrml97Parser.g
ANTLR Parser Generator   Version 2.7.4   1989-2004 jGuru.com
/usr/bin/antlr -o ../../../src/libopenvrml/openvrml -glib 
../../../src/libopenvrml/openvrml/Vrml97Parser.g 
../../../src/libopenvrml/openvrml/X3DVrmlParser.g
ANTLR Parser Generator   Version 2.7.4   1989-2004 jGuru.com
make[4]: Leaving directory 
`/home/braden/src/openvrml/openvrml/build/src/libopenvrml'

> Perhaps you could use a suffix rule
> 
> antlr.c:.
>   custom commands
> 
> and then add the antlr grammar file to *_SOURCES:
> 
> myprog_SOURCES = ... foo.antlr ...

I think the particulars of invoking antlr are such that a suffix rule
won't work.

Could the problem be related to the fact that I'm generating the sources
in $(srcdir)? I'm doing that because I want to distribute them: I don't
want users to have to rebuild them.

Note also that my generated sources do not get listed in *_SOURCES. They
are not compiled individually; rather, they are included by
openvrml/browser.cpp.

> Happy hacking in the New Year!

You too!

-- 
Braden McDaniel   e-mail: <[EMAIL PROTECTED]>
Jabber: <[EMAIL PROTECTED]>

Re: Built sources always regenerated

[Stepan Kasal]

Hello,

On Sat, Dec 31, 2005 at 02:51:30AM -0500, Braden McDaniel wrote:
> but the behavior is the same: the sources are regenerated every time,
> regardless of whether the antlr grammar has been modified.

I can imagine several problems:
1) there is a prerequisite which is always refreshed and causes this
2) If you use non-GNU make, it can may not understand that "./foo.c" and
"foo.c" is the same file.
3) If you work on a networked drive, the target may have time stamp in the
future, and make decides to refresh it.

I cannot guess more without seeing the makefile.  Could you please post
(an URL to) a self-contained example?  If you are able to cut some parts
and make the example smaller, that would definitely help.


> I have some sources that are generated with antlr. I have custom make
> rules that generate the sources from the antlr grammar. I have tried
> listing the generated sources in both BUILT_SOURCES and _DEPENDENCIES,

Overriding *_DEPENDENCIES is dangerous, it is better to let Automake
compute it.

The BUILT_SOURCES hack is useful sometimes, but it's better to avoid it.

Perhaps you could use a suffix rule

.antlr.c:
custom commands

and then add the antlr grammar file to *_SOURCES:

myprog_SOURCES = ... foo.antlr ...

Happy hacking in the New Year!
Stepan Kasal

Built sources always regenerated

[Braden McDaniel]

I have some sources that are generated with antlr. I have custom make
rules that generate the sources from the antlr grammar. I have tried
listing the generated sources in both BUILT_SOURCES and _DEPENDENCIES,
but the behavior is the same: the sources are regenerated every time,
regardless of whether the antlr grammar has been modified.

How can I keep this stuff from being regenerated all the time?

-- 
Braden McDaniel   e-mail: <[EMAIL PROTECTED]>
Jabber: <[EMAIL PROTECTED]>