Re: [CMake] Handling generated headers

[Michael Ellery]

> On Mar 27, 2017, at 2:57 PM, Bruce Stephens  
> wrote:
> 
> Yes, that's the idea: I have custom commands (created with
> add_custom_command) listing the headers in OUTPUT.
> 
> And there are source files which #include such headers.
> 
> I'd like it so that compiling such a source file would cause the
> header to be generated.
> 
> Concretely, with a CMakeLists.txt like the attached, with the obvious
> main.c (also attached), I'd like "ninja exe" to work.
> 
> I find that it does not.
> 
> After "ninja build-incl" (or "ninja incl.h") then "ninja exe" will
> work. And then after removing incl.h,
> "ninja exe" will recreate incl.h and then rebuild exe. (Presumably
> ninja is caching some dependency information, likely
> in its .ninja_deps file.) But correct recompilation (while obviously
> desirable) doesn't seem sufficient; I want the first
> build to work too.
> 
> (I'm also not sure why the add_custom_target is required, but it does
> seem to be. Presumably without it CMake
> can't see why the custom command might be used.)
> 
> In most cases I have a natural place where I need the things to be
> built, so the custom target can have ALL DEPENDS
> (rather than.the DEPENDS in the example). However, in some cases in
> parallel builds I still end up with (sometimes)
> source files not compiling because the needed headers haven't been generated.
> 
> As I say, so far this is OK: I can just generated dependencies
> explicitly in a pre-CMake step. But handling cases where
> main.c includes some source header which includes this generated
> incl.h seems not easily doable.
> 
> If I'm not missing something silly and all this is expected behaviour
> (as it appears to be) it's not a disaster. We can add
> a few explicit dependencies and get things to work. But maybe I'm
> missing something and this should work more
> smoothly.
> 
> 
> On Mon, Mar 27, 2017 at 4:08 PM, Michael Ellery  wrote:
> 

Doing this worked for me:


add_executable(exe main.c ${CMAKE_CURRENT_BINARY_DIR}/incl.h)

target_include_directories(exe PUBLIC ${CMAKE_CURRENT_BINARY_DIR})

add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/incl.h
COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_BINARY_DIR}/incl.h)


..in other words, I used the binary dir as the output location of the generated 
file and then I explicitly added the header file to the sources list. Yeah, it 
would be nice if header reps auto detection worked in this case, but it seems 
like it does not, so you form the dependency relationship explicitly I guess.

-Mike


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] Handling generated headers

[Bruce Stephens]

Yes, that's the idea: I have custom commands (created with
add_custom_command) listing the headers in OUTPUT.

And there are source files which #include such headers.

I'd like it so that compiling such a source file would cause the
header to be generated.

Concretely, with a CMakeLists.txt like the attached, with the obvious
main.c (also attached), I'd like "ninja exe" to work.

I find that it does not.

After "ninja build-incl" (or "ninja incl.h") then "ninja exe" will
work. And then after removing incl.h,
"ninja exe" will recreate incl.h and then rebuild exe. (Presumably
ninja is caching some dependency information, likely
in its .ninja_deps file.) But correct recompilation (while obviously
desirable) doesn't seem sufficient; I want the first
build to work too.

(I'm also not sure why the add_custom_target is required, but it does
seem to be. Presumably without it CMake
can't see why the custom command might be used.)

In most cases I have a natural place where I need the things to be
built, so the custom target can have ALL DEPENDS
(rather than.the DEPENDS in the example). However, in some cases in
parallel builds I still end up with (sometimes)
source files not compiling because the needed headers haven't been generated.

As I say, so far this is OK: I can just generated dependencies
explicitly in a pre-CMake step. But handling cases where
main.c includes some source header which includes this generated
incl.h seems not easily doable.

If I'm not missing something silly and all this is expected behaviour
(as it appears to be) it's not a disaster. We can add
a few explicit dependencies and get things to work. But maybe I'm
missing something and this should work more
smoothly.


On Mon, Mar 27, 2017 at 4:08 PM, Michael Ellery  wrote:
>
>> On Mar 27, 2017, at 6:31 AM, Bruce Stephens  
>> wrote:
>>
>> I have a build with two or three tools that generate headers and
>> source files. Getting the source files compiled is easy enough: when
>> they're mentioned as source files (in add_library or add_executable)
>> the custom rule gets triggered.
>>
>> But that doesn't seem to be true for header files included by
>> non-generated source files, presumably because CMake's not looking at
>> those files, but rather Ninja is (presumably other generators will
>> behave similarly). If the (generated) header changes then anything
>> using
>> the source file gets rebuilt (as expected), but the first build seems
>> not to necessarily succeed (depending on accidents of when the
>> generated files are produced).
>>
>> Concretely, suppose I have an XML file compat.xml with a custom rule
>> to generate messages/Compat.h, and some source files which include
>> that header (which does not exist to begin with), I'd like a compile
>> of any of those source files to depend on messages/Compat.h such that
>> it'll be generated if necessary.
>>
>> That doesn't seem to happen automatically using the Ninja generator.
>> Is that expected, or have I messed something up?
>>
>> For the moment I'm assuming it's as expected and have a workaround: a
>> simple Python script that scans source files and creates a deps.cmake
>> with calls
>> to add_file_dependencies. (Fortunately all the generated headers have
>> simple patterns so accurately determining them is straightforward.)
>>
>> Unfortunately there are also (non-generated) header files which
>> include these headers and that seems harder to handle.
>> Calling add_file_dependencies on header files doesn't seem to work.
>>
>> It's not a showstopper: I can just explicitly add some dependencies.
>> And later, we'll probably split out some of these things so they'll
>> be separate builds. It seems a bit annoying, though. Am I missing some
>> better way to handle this?
>> --
>
>
> it’s not clear from your message — are you using add_custom_command() to 
> create the rule that generates the header from the XML? If you get all of the 
> parameters right to that command (like OUTPUT, DEPENDS, BYPRODUCTS..), then I 
> would expect this to work and the header should get generated before any 
> other source that needs it. In some cases you might need to explicitly 
> specify that a target depends on that header, but if at least ONE thing 
> depends on the header, the the add_custom_command rule should get executed.  
> You might also try running CMAKE with —trace and generating a verbose 
> makefile if you are still not seeing your custom command running when it 
> should.
>
> -Mike Ellery
cmake_minimum_required(VERSION 3.7)
add_executable(exe main.c)
target_include_directories(exe PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
add_custom_command(OUTPUT incl.h
  COMMAND ${CMAKE_COMMAND} -E touch incl.h)
add_custom_target(build-incl DEPENDS incl.h)
#include "incl.h"
int main() {}
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake 

Re: [CMake] Handling generated headers

[Michael Ellery]

> On Mar 27, 2017, at 6:31 AM, Bruce Stephens  
> wrote:
> 
> I have a build with two or three tools that generate headers and
> source files. Getting the source files compiled is easy enough: when
> they're mentioned as source files (in add_library or add_executable)
> the custom rule gets triggered.
> 
> But that doesn't seem to be true for header files included by
> non-generated source files, presumably because CMake's not looking at
> those files, but rather Ninja is (presumably other generators will
> behave similarly). If the (generated) header changes then anything
> using
> the source file gets rebuilt (as expected), but the first build seems
> not to necessarily succeed (depending on accidents of when the
> generated files are produced).
> 
> Concretely, suppose I have an XML file compat.xml with a custom rule
> to generate messages/Compat.h, and some source files which include
> that header (which does not exist to begin with), I'd like a compile
> of any of those source files to depend on messages/Compat.h such that
> it'll be generated if necessary.
> 
> That doesn't seem to happen automatically using the Ninja generator.
> Is that expected, or have I messed something up?
> 
> For the moment I'm assuming it's as expected and have a workaround: a
> simple Python script that scans source files and creates a deps.cmake
> with calls
> to add_file_dependencies. (Fortunately all the generated headers have
> simple patterns so accurately determining them is straightforward.)
> 
> Unfortunately there are also (non-generated) header files which
> include these headers and that seems harder to handle.
> Calling add_file_dependencies on header files doesn't seem to work.
> 
> It's not a showstopper: I can just explicitly add some dependencies.
> And later, we'll probably split out some of these things so they'll
> be separate builds. It seems a bit annoying, though. Am I missing some
> better way to handle this?
> -- 


it’s not clear from your message — are you using add_custom_command() to create 
the rule that generates the header from the XML? If you get all of the 
parameters right to that command (like OUTPUT, DEPENDS, BYPRODUCTS..), then I 
would expect this to work and the header should get generated before any other 
source that needs it. In some cases you might need to explicitly specify that a 
target depends on that header, but if at least ONE thing depends on the header, 
the the add_custom_command rule should get executed.  You might also try 
running CMAKE with —trace and generating a verbose makefile if you are still 
not seeing your custom command running when it should.

-Mike Ellery
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake