Re: [cmake-developers] [CMake] add_custom_command() OUTPUT does not accept generator expressions, why?

[Yves Frederix]

>   add_library(test_lib STATIC $/generated_file.cpp)
>   add_custom_command(OUTPUT $/generated_file.cpp 
> ...)
>
> When we compute the output information like target file directory and
> target file name we need to also determine the linker language.  For
> that we need the full list of source files to infer the language.  If
> that list of sources uses generator expressions that refer back to the
> location of the target itself then we have a circular dependency in the
> computation.  This will have to be diagnosed and rejected.

Yes, agree. Or alternatively, document that source files that contain
generator expressions are not taken into account during automatic
linker language computation? In this case, you would need to set it
manually if this particular source file would make a difference.


>> I don't know anything about XCode, unfortunately. However,  Visual Studio
>> definitely supports a form of varying the source file list of the project
>> between configurations. That's because a source file can be marked as
>> _excluded_ from build for a particular configuration. So the solution
>> CMake could use would be to add all source files from all configurations
>> into the project, but mark them as excluded from [in]appropriate 
>> configurations.
>
> The current limitation is mostly because no one has thoroughly
> investigated how to support it in VS IDE projects.  If there is an
> approach that will work then we'd be happy to see it implemented.
> Similarly for Xcode.

In order to implement this, CMake should be able to determine if the
result of resolving a generator expression depends on the
configuration or not. Does the generator expression logic currently
allow such check? I did see
cmGeneratorExpression::GetHadContextSensitiveCondition(), but I have
the feeling it does not really do exactly that.

> A fallback approach could be to generate a wrapper source file
> that uses the preprocessor to conditionally include the real
> per-config source file based on the configuration.

That would be an option as well. I suspect best by using the
CMAKE_INTDIR preprocessor define?

Yves
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] add_custom_command() OUTPUT does not accept generator expressions, why?

[Brad King]

On 04/06/2016 08:02 AM, Petr Kmoch wrote:
> On 6 April 2016 at 09:51, Yves Frederix wrote:
>>   add_custom_target(test_target ALL SOURCES 
>> $/generated_file.cpp)

Watch out for circular logic here.  For example:

  add_library(test_lib STATIC $/generated_file.cpp)
  add_custom_command(OUTPUT $/generated_file.cpp ...)

When we compute the output information like target file directory and
target file name we need to also determine the linker language.  For
that we need the full list of source files to infer the language.  If
that list of sources uses generator expressions that refer back to the
location of the target itself then we have a circular dependency in the
computation.  This will have to be diagnosed and rejected.

>>  Target "test_target" has source files which vary by configuration.  This is
>>  not supported by the "Visual Studio 14 2015" generator.
>> 
>> Intuitively I understand why this is not supported (different
>> configurations share the same project file in VS, so which source file
>> would the project point to?) Is this also a technical limitation
>> though? Or do VS project files (and other multiconfig generator
>> project files such as xcode) support this setup but was it merely not
>> implemented yet?
> 
> I don't know anything about XCode, unfortunately. However,  Visual Studio
> definitely supports a form of varying the source file list of the project
> between configurations. That's because a source file can be marked as
> _excluded_ from build for a particular configuration. So the solution
> CMake could use would be to add all source files from all configurations
> into the project, but mark them as excluded from [in]appropriate 
> configurations.

The current limitation is mostly because no one has thoroughly
investigated how to support it in VS IDE projects.  If there is an
approach that will work then we'd be happy to see it implemented.
Similarly for Xcode.

A fallback approach could be to generate a wrapper source file
that uses the preprocessor to conditionally include the real
per-config source file based on the configuration.

-Brad

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] add_custom_command() OUTPUT does not accept generator expressions, why?

[Petr Kmoch]

On 6 April 2016 at 09:51, Yves Frederix 
wrote:

> [...]
> So, basically, a prerequisite for the above
> to work is that CMake behaves nicely for the following as well:
>
>   add_library(test_lib STATIC existing_file_$.cpp)
>
> At this moment CMake supports generator expressions in the source
> list, so this "should" work. It does not in practice, however:
>
>   CMake Error in CMakeLists.txt:
> Target "test_target" has source files which vary by configuration.
> This is
> not supported by the "Visual Studio 14 2015" generator.
>
> Intuitively I understand why this is not supported (different
> configurations share the same project file in VS, so which source file
> would the project point to?) Is this also a technical limitation
> though? Or do VS project files (and other multiconfig generator
> project files such as xcode) support this setup but was it merely not
> implemented yet?
>

I don't know anything about XCode, unfortunately. However,  Visual Studio
definitely supports a form of varying the source file list of the project
between configurations. That's because a source file can be marked as
_excluded_ from build for a particular configuration. So the solution CMake
could use would be to add all source files from all configurations into the
project, but mark them as excluded from [in]appropriate configurations.

Petr
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] add_custom_command() OUTPUT does not accept generator expressions, why?

[Yves Frederix]

Hi Brad,

Thanks for the pointers. I played with this a bit over the last week,
mainly to get familiar with the CMake internals, and it seems that
there are some things that might need discussion before getting to a
solution. Note that all my experiments were done using the VS
generator.

What we basically want to be able to do is something like:

  # Create any library.
  add_library(test_lib STATIC existing_file.cpp)
  # Generate a file in the same directory as the test_lib library was created.
  add_custom_target(test_target ALL SOURCES
$/generated_file.cpp)
  add_custom_command(
OUTPUT $/generated_file.cpp
COMMAND cmake -E touch "$/generated_file.cpp"
)

Without generator expressions, this works nicely. However, if CMake
were to correctly interpret generator expressions in OUTPUT, then the
corresponding source file would be
$/generated_file.cpp in add_custom_target,
which in turn, for a multiconfig generator would contain the
configuration in its path. So, basically, a prerequisite for the above
to work is that CMake behaves nicely for the following as well:

  add_library(test_lib STATIC existing_file_$.cpp)

At this moment CMake supports generator expressions in the source
list, so this "should" work. It does not in practice, however:

  CMake Error in CMakeLists.txt:
Target "test_target" has source files which vary by configuration.  This is
not supported by the "Visual Studio 14 2015" generator.

Intuitively I understand why this is not supported (different
configurations share the same project file in VS, so which source file
would the project point to?) Is this also a technical limitation
though? Or do VS project files (and other multiconfig generator
project files such as xcode) support this setup but was it merely not
implemented yet?

Yves



On Thu, Mar 31, 2016 at 3:39 PM, Brad King  wrote:
> On 03/31/2016 07:26 AM, Yves Frederix wrote:
>>> If anyone is interested in trying to implement generator expressions
>>> for custom command outputs, I can provide more details to get started.
>>
>> I am interested in having a go at this. I recently ran into this issue
>> at work as well and actually tried some things already. However, I
>> realized it was not that straightforward (not even talking about doing
>> it 'right' here ;)). Any pointers are welcome!
>
> Thanks!
>
> The main challenge is shown in the cmTargetTraceDependencies::FollowName
> method where it calls cmMakefile::GetSourceFileWithOutput.  This logic
> is done without knowledge of the current configuration.  Therefore we
> cannot expand generator expressions of the available outputs because
> they may contain $.  Some table of pre-expanded per-config
> outputs mapping back to their cmSourceFile instances may need to be
> built.  Achieving this may even require adding a generate-time
> companion of cmSourceFile that knows its configuration.  It will need
> to be investigated further.
>
> cmTargetTraceDependencies::CheckCustomCommand currently has a loop
> over all configurations to generate the union of dependencies.  This
> is used to allow generator expressions in the DEPENDS option.
> The cmTargetTraceDependencies logic may need to be refactored to move
> the loop over the configurations out to a higher level.  We may need
> to trace dependencies separately for each configuration (for multi-
> config generators).
>
> Also see cmCustomCommandGenerator.  The generators will need to
> be adapted to use this to get the custom command outputs with
> generator expressions expanded.  They already do for the commands
> and dependencies.
>
> -Brad
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] add_custom_command() OUTPUT does not accept generator expressions, why?

[Brad King]

On 03/31/2016 07:26 AM, Yves Frederix wrote:
>> If anyone is interested in trying to implement generator expressions
>> for custom command outputs, I can provide more details to get started.
> 
> I am interested in having a go at this. I recently ran into this issue
> at work as well and actually tried some things already. However, I
> realized it was not that straightforward (not even talking about doing
> it 'right' here ;)). Any pointers are welcome!

Thanks!

The main challenge is shown in the cmTargetTraceDependencies::FollowName
method where it calls cmMakefile::GetSourceFileWithOutput.  This logic
is done without knowledge of the current configuration.  Therefore we
cannot expand generator expressions of the available outputs because
they may contain $.  Some table of pre-expanded per-config
outputs mapping back to their cmSourceFile instances may need to be
built.  Achieving this may even require adding a generate-time
companion of cmSourceFile that knows its configuration.  It will need
to be investigated further.

cmTargetTraceDependencies::CheckCustomCommand currently has a loop
over all configurations to generate the union of dependencies.  This
is used to allow generator expressions in the DEPENDS option.
The cmTargetTraceDependencies logic may need to be refactored to move
the loop over the configurations out to a higher level.  We may need
to trace dependencies separately for each configuration (for multi-
config generators).

Also see cmCustomCommandGenerator.  The generators will need to
be adapted to use this to get the custom command outputs with
generator expressions expanded.  They already do for the commands
and dependencies.

-Brad
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] add_custom_command() OUTPUT does not accept generator expressions, why?

[Dan Liew]

Hi,

On 28 March 2016 at 15:05, Brad King  wrote:
> On 03/27/2016 06:11 AM, Dan Liew wrote:
>> OUTPUT does not accept generator expressions, why?
>
> It hasn't been implemented.  At least at one time it would have been
> very hard to implement.  I'm not sure now because there has been a lot
> of refactoring since I last looked at it.
>
> There is some discussion here:
>
>  https://cmake.org/Bug/view.php?id=13840
>  https://cmake.org/Bug/view.php?id=12877#c28315
>
>> If I can't use generator expressions with ``OUTPUT``, how am I
>> supposed to output a file into the build directory of a target?
>
> Unfortunately it is not possible to do cleanly without fixing the
> above.  Approximations include:
>
> * Use a POST_BUILD command on the target instead.
>
> * Make the OUTPUT a timestamp file and generate the real output
>   as a side effect.
>
> If anyone is interested in trying to implement generator expressions
> for custom command outputs, I can provide more details to get started.

Thanks for the info Brad. I wish I had time to look into this but I
don't so for now I'll just work around it.

Dan.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers