Re: [CMake] How do you handle recursive dependencies in CMake

[hemant]

Thanks much, Tim!

 

Hemant

 

From: CMake  On Behalf Of Tim Rae
Sent: Tuesday, June 19, 2018 9:39 AM
To: cmake@cmake.org
Subject: Re: [CMake] How do you handle recursive dependencies in CMake

 

Hi all,

 

I spent a bit of time working on this issue the last few days, and since I 
didn't see a satisfactory answer in this thread I thought I'd resurrect the 
thread since it's one of the first google hits.

it seems that all you need to do to solve the problem of transitive 
dependencies is make your package config files call find_package() / 
find_dependency() for their dependencies. I've posted some example code on a 
similar thread on stack overflow:

 

 <https://stackoverflow.com/a/50929353/2125915> 
https://stackoverflow.com/a/50929353/2125915

 

This is what has worked for me, and I believe it solves the problems discussed 
in this thread. I have only tested it at shallow levels of recursion, but I 
think it works. Please let me know if there is any problem.

 

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Tim Rae]

Hi all,

I spent a bit of time working on this issue the last few days, and since I
didn't see a satisfactory answer in this thread I thought I'd resurrect the
thread since it's one of the first google hits.
it seems that all you need to do to solve the problem of transitive
dependencies is make your package config files call find_package() /
find_dependency() for their dependencies. I've posted some example code on
a similar thread on stack overflow:

https://stackoverflow.com/a/50929353/2125915

This is what has worked for me, and I believe it solves the problems
discussed in this thread. I have only tested it at shallow levels of
recursion, but I think it works. Please let me know if there is any problem.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Ruslan Baratov via CMake]

Hi,

On 30-Jun-16 16:01, Sven Baars wrote:

I don't think this is a solution to the problem, since then it seems
like the build will still fail for all our users unless they also build
all their packages from inside hunter.
What do you mean by "from inside hunter"? Users have to add 
'hunter_add_package(YourProject)' + 'find_package(YourProject CONFIG 
REQUIRED)' this is true. You want to avoid that?



  If I wanted it to work just for
myself I could just use hardcoded paths. The problem is that I want a
portable way such that users can build my projects without losing
dependency information.
Hunter will download all dependencies automatically. 
YourProjectConfig.cmake should contain all dependencies too so 
'find_package(YourProject CONFIG REQUIRED)' is enough.



  In the current situation they are forced to
copy-paste all of my CMakeLists.txt files and add their own stuff to
that. At least if I follow the guides on the wiki/the advice that I got
so far. That is, unless I misunderstand something, which I hope I do,
but no one pointed that out yet.

Sven

Ruslo



On 06/30/2016 02:41 PM, Nicholas Braden wrote:

If find_project is not enough, and ExternalProject's only problem is
build duplication, then I think it makes sense to consider a
CMake-based dependency manager such as hunter:
https://github.com/ruslo/hunter


On Thu, Jun 30, 2016 at 3:59 AM, Sven Baars  wrote:

This is a reply to the options that Ray gave. Here I will use the
package dependencies C -> B -> A{1,2}:

1)  The "ad-hoc" method I first mentioned by setting
CMAKE_LIBRARY_OUTPUT_DIRECTORY.

As far as I understand, this would mean that every user of all of the
different projects would have to be forced to use this, and would not be
allowed to "install" anything anywhere else, which doesn't seem nice.

2)  ExternalProject which will grab a repository and build it.

This will not work. One of the projects I use is Trilinos, which has
build of around 1GB. I don't want to pull and build that for every
project I have. Also the build flags that are used sometimes differ per
machine, not per project, so it would be nice if I could build it only
once per machine.

Also, in a more generalized sense, this would also mean that every
project I pull with ExtenalProject should also pull its own dependencies
with ExternalProject. So then if every project on my system used CMake,
this would mean that I would recursively rebuild my entire system for
every project I have. This doesn't seem right.

3)  Some Find_Package () mechanism that will do a search for it.

The point I had is that we actually try to use this. However, the
find_package does not find all dependencies. And we don't know in
package C whether it depends on A1 or A2, because of build flags/CMake
checks that were used for project B. So we can't just do a find_package
for either A1 or A2, because we don't know which one was used unless we
perform all the CMake checks that were done in project B (in some cases
10k+ lines of CMake code).

4)  Your option of including *.cmake files that provide the paths
[sorry, I might have misunderstood it].

This, so far, is the only option, because then B can tell us that it
used A2, not A1. This can just be done by providing absolute paths to
the libraries that were used in the compilation of B. But we are looking
for a standardized way to do this. I'd prefer to not have a lot of
custom code in all of my libraries.

Now some more information:

On supercomputers it is very common that every library on the system is
installed in a different directory. This is so every user can load their
own version of the library without breaking the system for others.
Therefore, you will never find libraries that are installed in the
standard system directories where CMake looks for the libraries. By
using PATH you can make it able to find the place where to look for the
FoobarConfig.cmake files, which is great when you want to build project
B, and this is also done automatically on all supercomputers I work on,
but those config files do not contain information on where the actual
libraries of project A are when you build project C. I guess Cfyz and me
think they should in some standardized way.

Sven
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers 

Re: [CMake] How do you handle recursive dependencies in CMake

[Alan W. Irwin]

On 2016-06-30 10:59+0200 Sven Baars wrote:


2)  ExternalProject which will grab a repository and build it.

This will not work. One of the projects I use is Trilinos, which has
build of around 1GB. I don't want to pull and build that for every
project I have. Also the build flags that are used sometimes differ per
machine, not per project, so it would be nice if I could build it only
once per machine.

Also, in a more generalized sense, this would also mean that every
project I pull with ExtenalProject should also pull its own dependencies
with ExternalProject. So then if every project on my system used CMake,
this would mean that I would recursively rebuild my entire system for
every project I have. This doesn't seem right.


Hi Sven:

I am chiming in late on this thread because I became aware of it only
recently.

My advice is to look again at ExternalProject because there are some
misconceptions above.  For example, in my epa_build superproject I use
ExternalProject to build a whole set of different projects (some with
CMake-based build systems some with autotools-based build system) that
interdepend on each other in a large tree of dependencies and which
also have external dependencies that must be installed on the machine
that are not controlled by my epa_build superproject. ExternalProject
allows you to set up dependencies between those epa_built projects so
they are built and installed only once and only if needed directly or
transitively by the particular epa_built "starter" project the user
specifies.  I have implemented a useful CMake function so that each
project that I make part of epa_build typically only needs to know
what other epa_built projects it depends on and the location of the
source code to be downloaded for the project.  So the epa_build
CMakeList.txt files that are required to describe each project tend to
be quite short.

For epa_build I do simplify my life by using two common install
locations (one for the build tools that are epa_built, one for the
libraries that are epa_built) for every project that I build and
install with epa_build.  So that vastly simplifies the find issues for
the dependencies which are epa_built projects themselves.  The
external dependencies (not under epa_build control) for each project
are a different story, but in my case those are largely system
dependencies found in the expected location.  So the result is I have
virtually no find issues at all with epa_build.

To learn more about epa_build see

or the latest release of PLplot.  (I originally set up epa_build so
that I could straightforwardly build the latest versions of PLplot
dependencies to test PLplot against those rather than being limited to
testing PLplot against dated system versions of its dependencies.  But
I now use it for any software that I like to build for myself.)

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey again Marcel,

Your reply got me thinking. I just checked again, and found that I just
have to set the LIBRARY_PATH correctly as well. After that everything
seems to work. Apparently this was not done in the right way. Since I
have to use PATH anyway to find the FoobarConfig.cmake files, it is just
as easy to also set LIBRARY_PATH to find the dependencies of the already
compiled libraries. The problem was that PATH was already set in the
right way, so I didn't notice that LIBRARY_PATH was not set correctly.

Anyhow, the thing that ultimately confused me in such a way that I
thought the problem was somewhere else was that

https://cmake.org/cmake/help/v3.0/command/link_directories.html

says that find_package should return *absolute* paths, which it does
not, since it just returns the targets. Maybe that can be cleared up a
bit to prevent further confusion.

Thanks a lot for pointing me in the right direction!

Cheers,
Sven
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey Marcel,

They are indeed always set correctly, and for this reason, find_package
is also always able to find these packages, which is great, and this is
why I love using CMake, but then it still happens that when you don't
explicitly use find_package, so in this case I don't use find_package(A)
in package C because C only depends on B, it will give an error, as can
be seen in my example on Github:

https://github.com/Sbte/cmake-example

The error is:

Linking CXX shared library libC.so
/usr/bin/ld: cannot find -lA

It might be that I am just doing something wrong here, so in that case
please let me know.

Cheers,
Sven
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Marcel Loose]

Hi Sven,

Sorry to chime in so late. I was wondering, since you're mentioning
large computer clusters, don't you use the "module" system to select
which versions of which packages you want to use? If so, then (I assume)
your PATH and LD_LIBRARY_PATH environment variables will have been set
such that the correct binaries and libraries can be found, right? CMake
also searches for libraries in the paths specified in the environment
variables PATH and LIB (unfortunately not LD_LIBRARY_PATH). Isn't that
the way to tackle this problem?

Cheers,
Marcel Loose.


On 30/06/16 13:42, Sven Baars wrote:
> Hey Ray,
>
> For myself it is not a big problem either, but I don't want to tell
> users of my libraries that they have to look up how the administrators
> at their supercomputing facility compiled 20 other libraries so they can
> set  their environment variables in the right way, while at the same
> time making my own CMakeLists.txt files much longer (because I would
> need many more find_package calls for dependencies of dependencies that
> I don't use). Whereas if CMake would just provide some means of using
> absolute paths instead of targets for libraries, or attach the
> information about those paths to the targets or something like that,
> this would never be a problem, since in that case everything would be
> found automatically.
>
> Also, it still doesn't make sense to me that instead of one find_package
> call for a package I actually use, I also need 20 find_package calls for
> dependencies of that other package. Also it doesn't make sense that I
> would actually need these find_package calls, whereas if the packages
> were just installed in /usr/lib or something it would work without any
> knowledge of this package even being used.
>
> Something that causes problems is the case where you use either A1 or
> A2. Then in B you would do something like
>
> option(B_USE_A2 OFF "Use package A2 instead of A1")
>
> if (NOT B_USE_A2)
> find_package(A1)
> if (A1_FOUND)
> ...
> target_link_libraries(B ${A1_LIBRARIES})
> ...
> else()
> set(B_USE_A2 ON)
> endif()
>
> if (B_USE_A2)
> find_package(A2)
> if (A2_FOUND)
> ...
> target_link_libraries(B ${A2_LIBRARIES})
> ...
> endif()
> endif()
>
> which makes sense, but then also in C instead of doing
>
> find_package(B)
> if (B_FOUND)
> target_link_libraries(C ${B_LIBRARIES})
> endif()
>
> you would also have to do
>
> if (B_FOUND)
> option(C_USE_A2 OFF "Use package A2 instead of A1, which is used in
> package B. Make sure C_USE_A2 is equal to what you used in package B as
> B_USE_A2")
>
> if (NOT C_USE_A2)
> find_package(A1)
> if (A1_FOUND)
> ...
> target_link_libraries(C ${A1_LIBRARIES})
> ...
> else()
> set(C_USE_A2 ON)
> endif()
>
> if (C_USE_A2)
> find_package(A2)
> if (A2_FOUND)
> ...
> target_link_libraries(C ${A2_LIBRARIES})
> ...
> endif()
> endif()
> target_link_libraries(C ${B_LIBRARIES})
> endif()
>
> or something like that, which is quite silly. Especially the part where
> compilation fails if B_USE_A2 is unequal to C_USE_A2. In general, my
> CMakeLists.txt is a lot more complicated, so I'd prefer the option where
> I only look for B.
>
> Sven
>
> On 06/30/2016 01:05 PM, Raymond Wan wrote:
>> Hi Sven,
>>
>>
>> On Thu, Jun 30, 2016 at 6:40 PM, Sven Baars  wrote:
>>> So let's take Trilinos as an example, which is quite a big library,
>>> which depends on a lot of libraries as well. In every project that
>>> builds upon it (but does not even use functionality of any libraries),
>>> we would have to set: SCALAPACK_ROOT, LAPACK_ROOT, BLAS_ROOT,
>>> BOOST_ROOT, GTEST_ROOT, BLACS_ROOT, UMFPACK_ROOT, MKL_ROOT, etc... and
>>> then inside the CMakeFiles.txt do a find_package(Trilinos),
>>> find_package(Scalapack), find_package(LAPACK), etc... just to compile
>>> int main() {} with one call to a Trilinos function inside of it. I'd say
>>> that it should be enough to just do a find_package(Trilinos).
>> Perhaps I'm alone here, but I actually don't see this as being a "big" 
>> problem.
>>
>> For the environment variables, you can place them all in a bash script
>> and source it during log in.
>>
>> For all of the Find_Package calls, they only matter whenever you have
>> to re-run cmake.  I don't know what Trilinos is, but for something
>> that big, I presume it takes more time to compile it (during active
>> development) than to run cmake to look through all those paths...
>>
>> Ray

<>

signature.asc
Description: OpenPGP digital signature
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: 

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

I don't think this is a solution to the problem, since then it seems
like the build will still fail for all our users unless they also build
all their packages from inside hunter. If I wanted it to work just for
myself I could just use hardcoded paths. The problem is that I want a
portable way such that users can build my projects without losing
dependency information. In the current situation they are forced to
copy-paste all of my CMakeLists.txt files and add their own stuff to
that. At least if I follow the guides on the wiki/the advice that I got
so far. That is, unless I misunderstand something, which I hope I do,
but no one pointed that out yet.

Sven

On 06/30/2016 02:41 PM, Nicholas Braden wrote:
> If find_project is not enough, and ExternalProject's only problem is
> build duplication, then I think it makes sense to consider a
> CMake-based dependency manager such as hunter:
> https://github.com/ruslo/hunter
>
>
> On Thu, Jun 30, 2016 at 3:59 AM, Sven Baars  wrote:
>> This is a reply to the options that Ray gave. Here I will use the
>> package dependencies C -> B -> A{1,2}:
>>
>> 1)  The "ad-hoc" method I first mentioned by setting
>> CMAKE_LIBRARY_OUTPUT_DIRECTORY.
>>
>> As far as I understand, this would mean that every user of all of the
>> different projects would have to be forced to use this, and would not be
>> allowed to "install" anything anywhere else, which doesn't seem nice.
>>
>> 2)  ExternalProject which will grab a repository and build it.
>>
>> This will not work. One of the projects I use is Trilinos, which has
>> build of around 1GB. I don't want to pull and build that for every
>> project I have. Also the build flags that are used sometimes differ per
>> machine, not per project, so it would be nice if I could build it only
>> once per machine.
>>
>> Also, in a more generalized sense, this would also mean that every
>> project I pull with ExtenalProject should also pull its own dependencies
>> with ExternalProject. So then if every project on my system used CMake,
>> this would mean that I would recursively rebuild my entire system for
>> every project I have. This doesn't seem right.
>>
>> 3)  Some Find_Package () mechanism that will do a search for it.
>>
>> The point I had is that we actually try to use this. However, the
>> find_package does not find all dependencies. And we don't know in
>> package C whether it depends on A1 or A2, because of build flags/CMake
>> checks that were used for project B. So we can't just do a find_package
>> for either A1 or A2, because we don't know which one was used unless we
>> perform all the CMake checks that were done in project B (in some cases
>> 10k+ lines of CMake code).
>>
>> 4)  Your option of including *.cmake files that provide the paths
>> [sorry, I might have misunderstood it].
>>
>> This, so far, is the only option, because then B can tell us that it
>> used A2, not A1. This can just be done by providing absolute paths to
>> the libraries that were used in the compilation of B. But we are looking
>> for a standardized way to do this. I'd prefer to not have a lot of
>> custom code in all of my libraries.
>>
>> Now some more information:
>>
>> On supercomputers it is very common that every library on the system is
>> installed in a different directory. This is so every user can load their
>> own version of the library without breaking the system for others.
>> Therefore, you will never find libraries that are installed in the
>> standard system directories where CMake looks for the libraries. By
>> using PATH you can make it able to find the place where to look for the
>> FoobarConfig.cmake files, which is great when you want to build project
>> B, and this is also done automatically on all supercomputers I work on,
>> but those config files do not contain information on where the actual
>> libraries of project A are when you build project C. I guess Cfyz and me
>> think they should in some standardized way.
>>
>> Sven
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at: 
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more 
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at 
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: 

Re: [CMake] How do you handle recursive dependencies in CMake

[Nicholas Braden]

If find_project is not enough, and ExternalProject's only problem is
build duplication, then I think it makes sense to consider a
CMake-based dependency manager such as hunter:
https://github.com/ruslo/hunter


On Thu, Jun 30, 2016 at 3:59 AM, Sven Baars  wrote:
> This is a reply to the options that Ray gave. Here I will use the
> package dependencies C -> B -> A{1,2}:
>
> 1)  The "ad-hoc" method I first mentioned by setting
> CMAKE_LIBRARY_OUTPUT_DIRECTORY.
>
> As far as I understand, this would mean that every user of all of the
> different projects would have to be forced to use this, and would not be
> allowed to "install" anything anywhere else, which doesn't seem nice.
>
> 2)  ExternalProject which will grab a repository and build it.
>
> This will not work. One of the projects I use is Trilinos, which has
> build of around 1GB. I don't want to pull and build that for every
> project I have. Also the build flags that are used sometimes differ per
> machine, not per project, so it would be nice if I could build it only
> once per machine.
>
> Also, in a more generalized sense, this would also mean that every
> project I pull with ExtenalProject should also pull its own dependencies
> with ExternalProject. So then if every project on my system used CMake,
> this would mean that I would recursively rebuild my entire system for
> every project I have. This doesn't seem right.
>
> 3)  Some Find_Package () mechanism that will do a search for it.
>
> The point I had is that we actually try to use this. However, the
> find_package does not find all dependencies. And we don't know in
> package C whether it depends on A1 or A2, because of build flags/CMake
> checks that were used for project B. So we can't just do a find_package
> for either A1 or A2, because we don't know which one was used unless we
> perform all the CMake checks that were done in project B (in some cases
> 10k+ lines of CMake code).
>
> 4)  Your option of including *.cmake files that provide the paths
> [sorry, I might have misunderstood it].
>
> This, so far, is the only option, because then B can tell us that it
> used A2, not A1. This can just be done by providing absolute paths to
> the libraries that were used in the compilation of B. But we are looking
> for a standardized way to do this. I'd prefer to not have a lot of
> custom code in all of my libraries.
>
> Now some more information:
>
> On supercomputers it is very common that every library on the system is
> installed in a different directory. This is so every user can load their
> own version of the library without breaking the system for others.
> Therefore, you will never find libraries that are installed in the
> standard system directories where CMake looks for the libraries. By
> using PATH you can make it able to find the place where to look for the
> FoobarConfig.cmake files, which is great when you want to build project
> B, and this is also done automatically on all supercomputers I work on,
> but those config files do not contain information on where the actual
> libraries of project A are when you build project C. I guess Cfyz and me
> think they should in some standardized way.
>
> Sven
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at: 
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more 
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at 
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey Ray,

For myself it is not a big problem either, but I don't want to tell
users of my libraries that they have to look up how the administrators
at their supercomputing facility compiled 20 other libraries so they can
set  their environment variables in the right way, while at the same
time making my own CMakeLists.txt files much longer (because I would
need many more find_package calls for dependencies of dependencies that
I don't use). Whereas if CMake would just provide some means of using
absolute paths instead of targets for libraries, or attach the
information about those paths to the targets or something like that,
this would never be a problem, since in that case everything would be
found automatically.

Also, it still doesn't make sense to me that instead of one find_package
call for a package I actually use, I also need 20 find_package calls for
dependencies of that other package. Also it doesn't make sense that I
would actually need these find_package calls, whereas if the packages
were just installed in /usr/lib or something it would work without any
knowledge of this package even being used.

Something that causes problems is the case where you use either A1 or
A2. Then in B you would do something like

option(B_USE_A2 OFF "Use package A2 instead of A1")

if (NOT B_USE_A2)
find_package(A1)
if (A1_FOUND)
...
target_link_libraries(B ${A1_LIBRARIES})
...
else()
set(B_USE_A2 ON)
endif()

if (B_USE_A2)
find_package(A2)
if (A2_FOUND)
...
target_link_libraries(B ${A2_LIBRARIES})
...
endif()
endif()

which makes sense, but then also in C instead of doing

find_package(B)
if (B_FOUND)
target_link_libraries(C ${B_LIBRARIES})
endif()

you would also have to do

if (B_FOUND)
option(C_USE_A2 OFF "Use package A2 instead of A1, which is used in
package B. Make sure C_USE_A2 is equal to what you used in package B as
B_USE_A2")

if (NOT C_USE_A2)
find_package(A1)
if (A1_FOUND)
...
target_link_libraries(C ${A1_LIBRARIES})
...
else()
set(C_USE_A2 ON)
endif()

if (C_USE_A2)
find_package(A2)
if (A2_FOUND)
...
target_link_libraries(C ${A2_LIBRARIES})
...
endif()
endif()
target_link_libraries(C ${B_LIBRARIES})
endif()

or something like that, which is quite silly. Especially the part where
compilation fails if B_USE_A2 is unequal to C_USE_A2. In general, my
CMakeLists.txt is a lot more complicated, so I'd prefer the option where
I only look for B.

Sven

On 06/30/2016 01:05 PM, Raymond Wan wrote:
> Hi Sven,
>
>
> On Thu, Jun 30, 2016 at 6:40 PM, Sven Baars  wrote:
>> So let's take Trilinos as an example, which is quite a big library,
>> which depends on a lot of libraries as well. In every project that
>> builds upon it (but does not even use functionality of any libraries),
>> we would have to set: SCALAPACK_ROOT, LAPACK_ROOT, BLAS_ROOT,
>> BOOST_ROOT, GTEST_ROOT, BLACS_ROOT, UMFPACK_ROOT, MKL_ROOT, etc... and
>> then inside the CMakeFiles.txt do a find_package(Trilinos),
>> find_package(Scalapack), find_package(LAPACK), etc... just to compile
>> int main() {} with one call to a Trilinos function inside of it. I'd say
>> that it should be enough to just do a find_package(Trilinos).
>
> Perhaps I'm alone here, but I actually don't see this as being a "big" 
> problem.
>
> For the environment variables, you can place them all in a bash script
> and source it during log in.
>
> For all of the Find_Package calls, they only matter whenever you have
> to re-run cmake.  I don't know what Trilinos is, but for something
> that big, I presume it takes more time to compile it (during active
> development) than to run cmake to look through all those paths...
>
> Ray

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Raymond Wan]

Hi Sven,


On Thu, Jun 30, 2016 at 6:40 PM, Sven Baars  wrote:
> So let's take Trilinos as an example, which is quite a big library,
> which depends on a lot of libraries as well. In every project that
> builds upon it (but does not even use functionality of any libraries),
> we would have to set: SCALAPACK_ROOT, LAPACK_ROOT, BLAS_ROOT,
> BOOST_ROOT, GTEST_ROOT, BLACS_ROOT, UMFPACK_ROOT, MKL_ROOT, etc... and
> then inside the CMakeFiles.txt do a find_package(Trilinos),
> find_package(Scalapack), find_package(LAPACK), etc... just to compile
> int main() {} with one call to a Trilinos function inside of it. I'd say
> that it should be enough to just do a find_package(Trilinos).


Perhaps I'm alone here, but I actually don't see this as being a "big" problem.

For the environment variables, you can place them all in a bash script
and source it during log in.

For all of the Find_Package calls, they only matter whenever you have
to re-run cmake.  I don't know what Trilinos is, but for something
that big, I presume it takes more time to compile it (during active
development) than to run cmake to look through all those paths...

Ray
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hi Ray,

This still has the same problem. If I use FindBoost in project B, I can
use BOOST_ROOT to be able to find it. This works fine. Now, if I try to
compile project C, I can set B_ROOT for FindB, but then Boost will not
be found, because it is not in B_ROOT, so project C will not compile.
That is, unless Boost sets absolute paths for all its libraries. Which
is something we have would have to do manually with custom CMake code as
per option 4.

I don't find it weird that CMake should be able to find Boost in project
C, since Boost was already found and used in project B. And we only link
to project B explicitly in project C, not to Boost. Actually in C we
don't use any Boost functionality other than what is used inside of B.
So why should C then explicitly look for Boost, require BOOST_ROOT to do
so, then do a lot of checks to see if B actually uses Boost, and in that
case link to it, when we don't use Boost in C?

So let's take Trilinos as an example, which is quite a big library,
which depends on a lot of libraries as well. In every project that
builds upon it (but does not even use functionality of any libraries),
we would have to set: SCALAPACK_ROOT, LAPACK_ROOT, BLAS_ROOT,
BOOST_ROOT, GTEST_ROOT, BLACS_ROOT, UMFPACK_ROOT, MKL_ROOT, etc... and
then inside the CMakeFiles.txt do a find_package(Trilinos),
find_package(Scalapack), find_package(LAPACK), etc... just to compile
int main() {} with one call to a Trilinos function inside of it. I'd say
that it should be enough to just do a find_package(Trilinos).

Note that by finding a library, I mean that it should know where it is,
by for instance using absolute paths at linking time, which I think is
sufficient. I don't mean doing all the things that are done in Find*, or
going through directories to find the library.

Sven

On 06/30/2016 12:10 PM, Raymond Wan wrote:
> Hi Sven,
>
>
> On Thu, Jun 30, 2016 at 4:59 PM, Sven Baars  wrote:
>> This is a reply to the options that Ray gave. Here I will use the
>> package dependencies C -> B -> A{1,2}:
>
> Well, I presented some options to you.  Perhaps there are others, but
> if none of them are right for you then there's nothing more I can add,
> I think.
>
>
>> On supercomputers it is very common that every library on the system is
>> installed in a different directory. This is so every user can load their
>> own version of the library without breaking the system for others.
>
> Sure.  So, during the CMake run, you can set CMAKE_INSTALL_PREFIX to
> be something like ~/lib/ or ~/mylib/ or whatever.
>
>
>> Therefore, you will never find libraries that are installed in the
>> standard system directories where CMake looks for the libraries. By
>
> Well, I wasn't suggesting that you use Find_Package as-is.  I mean the
> phrase "CMake looks for the libraries" isn't quite correct here since
> its behaviour is affected by the Find_Package's implementation.
>
> Presumably, you will need to write your own for the modules you're
> working with.  One person would write it and this file would be
> shared.
>
> How it would work is "something like" (i.e., you need to look at how
> it works yourself) the Boost library.  With that, you can set
> BOOST_ROOT to override the system-installed version.  (Take a look at:
>  https://cmake.org/cmake/help/v3.3/module/FindBoost.html)
>
> I think I said this early on, but perhaps I didn't explain it very well.
>
> Thus, each user would set an environment variable that would affect
> the search path of this Find_Package that you wrote.  If this is
> troublesome, then you can add some favourite paths like ~/lib/ and
> ~/mylib/ as default paths to the search first.
>
> I think it is a bit unreasonable for CMake to "automatically" know
> where libraries are stored for each user other than performing a
> recursive search down ~/ .  (I suppose that's possible.)  To reduce
> the search space for binary programs that we've installed in ~/, we
> would set the PATH variable.  I don't think it's that much different
> from setting something like BOOST_ROOT and then getting a Find_Package
> that you've written to use this as the starting point for a search.
>
> I mean, it seems strange to expect CMake to be able to find archive
> files by itself but, by default, we do not expect the same for
> executable files...
>
> Ray

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Raymond Wan]

Hi Sven,


On Thu, Jun 30, 2016 at 4:59 PM, Sven Baars  wrote:
> This is a reply to the options that Ray gave. Here I will use the
> package dependencies C -> B -> A{1,2}:


Well, I presented some options to you.  Perhaps there are others, but
if none of them are right for you then there's nothing more I can add,
I think.


> On supercomputers it is very common that every library on the system is
> installed in a different directory. This is so every user can load their
> own version of the library without breaking the system for others.


Sure.  So, during the CMake run, you can set CMAKE_INSTALL_PREFIX to
be something like ~/lib/ or ~/mylib/ or whatever.


> Therefore, you will never find libraries that are installed in the
> standard system directories where CMake looks for the libraries. By


Well, I wasn't suggesting that you use Find_Package as-is.  I mean the
phrase "CMake looks for the libraries" isn't quite correct here since
its behaviour is affected by the Find_Package's implementation.

Presumably, you will need to write your own for the modules you're
working with.  One person would write it and this file would be
shared.

How it would work is "something like" (i.e., you need to look at how
it works yourself) the Boost library.  With that, you can set
BOOST_ROOT to override the system-installed version.  (Take a look at:
 https://cmake.org/cmake/help/v3.3/module/FindBoost.html)

I think I said this early on, but perhaps I didn't explain it very well.

Thus, each user would set an environment variable that would affect
the search path of this Find_Package that you wrote.  If this is
troublesome, then you can add some favourite paths like ~/lib/ and
~/mylib/ as default paths to the search first.

I think it is a bit unreasonable for CMake to "automatically" know
where libraries are stored for each user other than performing a
recursive search down ~/ .  (I suppose that's possible.)  To reduce
the search space for binary programs that we've installed in ~/, we
would set the PATH variable.  I don't think it's that much different
from setting something like BOOST_ROOT and then getting a Find_Package
that you've written to use this as the starting point for a search.

I mean, it seems strange to expect CMake to be able to find archive
files by itself but, by default, we do not expect the same for
executable files...

Ray
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

This is a reply to the options that Ray gave. Here I will use the
package dependencies C -> B -> A{1,2}:

1)  The "ad-hoc" method I first mentioned by setting
CMAKE_LIBRARY_OUTPUT_DIRECTORY.

As far as I understand, this would mean that every user of all of the
different projects would have to be forced to use this, and would not be
allowed to "install" anything anywhere else, which doesn't seem nice.

2)  ExternalProject which will grab a repository and build it.

This will not work. One of the projects I use is Trilinos, which has
build of around 1GB. I don't want to pull and build that for every
project I have. Also the build flags that are used sometimes differ per
machine, not per project, so it would be nice if I could build it only
once per machine.

Also, in a more generalized sense, this would also mean that every
project I pull with ExtenalProject should also pull its own dependencies
with ExternalProject. So then if every project on my system used CMake,
this would mean that I would recursively rebuild my entire system for
every project I have. This doesn't seem right.

3)  Some Find_Package () mechanism that will do a search for it.

The point I had is that we actually try to use this. However, the
find_package does not find all dependencies. And we don't know in
package C whether it depends on A1 or A2, because of build flags/CMake
checks that were used for project B. So we can't just do a find_package
for either A1 or A2, because we don't know which one was used unless we
perform all the CMake checks that were done in project B (in some cases
10k+ lines of CMake code).

4)  Your option of including *.cmake files that provide the paths
[sorry, I might have misunderstood it].

This, so far, is the only option, because then B can tell us that it
used A2, not A1. This can just be done by providing absolute paths to
the libraries that were used in the compilation of B. But we are looking
for a standardized way to do this. I'd prefer to not have a lot of
custom code in all of my libraries.

Now some more information:

On supercomputers it is very common that every library on the system is
installed in a different directory. This is so every user can load their
own version of the library without breaking the system for others.
Therefore, you will never find libraries that are installed in the
standard system directories where CMake looks for the libraries. By
using PATH you can make it able to find the place where to look for the
FoobarConfig.cmake files, which is great when you want to build project
B, and this is also done automatically on all supercomputers I work on,
but those config files do not contain information on where the actual
libraries of project A are when you build project C. I guess Cfyz and me
think they should in some standardized way.

Sven
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Cfyz]

On 29 June 2016 at 05:08, Raymond Wan  wrote:
> I agree that what we're observing is a problem, but I don't think it is
that big.

I must admit I am mixing a bit of my own situation here and trying to think
about the most general case. It's just there is a missing bit in the
dependency search system and how far will it bleed into a project depends
on the situation. In my case it breaks on static libraries. Neither
installing in same stage directory nor ExternalProject go around the fact
that such targets just does not provide enough to link against them.

In case of dynamic libraries, it breaks if components are not located in
(or installed to during build) the same path. Library B may find its
dependency with some help (e. g. LIB_A_PATH during configure) and list
libA.so in the dynamic section. But the next steps will still need to know
libA.so location to link.


On 29 June 2016 at 05:08, Raymond Wan  wrote:
> ExternalProject which will grab a repository and build it.

Still, what about the fact that ExternalProject provides dependency
information on the main project build step, after the configuration? I've
found it quite disappointing and severely limiting ExternalProject
usefulness. Especially with CMake-based projects where you cannot use usual
find_package to search for that external project because its configuration
file would not be installed until much later.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Raymond Wan]

On Wed, Jun 29, 2016 at 3:23 AM, Cfyz  wrote:
> On 28 June 2016 at 17:19, Raymond Wan  wrote:
>> I think if I were presented with the same problem as Sven, I'd <...> Or,
>> if that wasn't possible, use ExternalProject.
>> It's inefficient in terms of disk space, but at least A (assuming A
>> depends on B) will be built correctly.
> What I'm trying to solve (and what Sven's problem looks very similar to) is
> more about keeping the information about dependencies. You've mentioned that
> yourself in the linked discussion: the real problem are projects with
> complex tree-like dependencies, more than one edge deep. And once those
> dependencies become optional and configurable, things break completely. It
> is impossible to search for the dependency you don't even know about =(.


I agree that what we're observing is a problem, but I don't think it
is that big.  There are at least 3 solutions discussed so far:

1)  The "ad-hoc" method I first mentioned by setting
CMAKE_LIBRARY_OUTPUT_DIRECTORY.
2)  ExternalProject which will grab a repository and build it.
3)  Some Find_Package () mechanism that will do a search for it.
4)  Your option of including *.cmake files that provide the paths
[sorry, I might have misunderstood it].

If we have some kind of transitive dependency:  A depends on B; B depends on C.

Then when we build A, we don't really need to know [in detail] that
this transitive dependency exists (*).  All the compiler needs to know
is where is B and where is C.  And if B and C are in the same
directory (i.e., a system-level directory), that's fine.

Suppose B and C are in two different directories.  I don't know if
it's common, but often, I see install scripts provide a symbolic link
from the system-level directory to where B and C are

In Sven's case (sorry to put you on the spot, Sven!  :-) ), the 3
options don't work because:

1)  Modules are in different source trees.
2)  Duplication between users on a super-computer.
3)  Too many combinations of Find_Package ().
4)  Seems ad-hoc?

Though not perfect, either (2) or (3) could solve Sven's problem.  For
(2), ignore duplication.  For (3), how about something like what
Find_Boost does?  It searches for system-level directories, but you
can nudge it by setting an environment variable (i.e., BOOST_ROOT)
where it will search first.  This means each user will have to set an
environment variable (or set it on the command-line of CMake with -D).

Hm, I *think* any of this is fine (for Sven's problem), but if it
isn't then perhaps I'm misunderstanding the problem...

Ray

(*)  At least, the order to the archives in target_link_libraries have
to be correct.  But that is done once by the CMakeLists.txt writer and
not something the end user needs to worry about.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Cfyz]

Hello Sven, Raymond.

On 28 June 2016 at 16:07, Sven Baars  wrote:
> Hence my question if there is some "right" way of doing this with CMake,
without all of the code I wrote to work around the problem.

I am under impression there is an important bit missing from CMake export
system. If some library has an unspecified set of dependencies (decided
during its configuration) then it will not be able to generate a config
file for find_package with CMake default tools alone. I think I will raise
it as an issue at the main CMake issue tracker after some more
investigation.


On 28 June 2016 at 16:07, Sven Baars  wrote:
> I also wrote a lot of homebrew code you mentioned at the end of the post,
but it indeed always seems to always break something for some user of the
code
> In the mean time I'll have a look at the code you wrote, but it seems
very similar to something I already had, which broke for some users.

The code examples and what I've mentioned earlier are just pieces of the
possible solution. A complete export has a lot of nuances (e. g.
versioning, build configurations, etc.) and is unlikely to work flawlessly
right from the start. But think it might be worthwhile to try and make such
helper module somewhere on github.


On 28 June 2016 at 17:19, Raymond Wan  wrote:
> I think if I were presented with the same problem as Sven, I'd <...> Or,
if that wasn't possible, use ExternalProject.
> It's inefficient in terms of disk space, but at least A (assuming A
depends on B) will be built correctly.

What I'm trying to solve (and what Sven's problem looks very similar to) is
more about keeping the information about dependencies. You've mentioned
that yourself in the linked discussion: the real problem are projects with
complex tree-like dependencies, more than one edge deep. And once those
dependencies become optional and configurable, things break completely. It
is impossible to search for the dependency you don't even know about =(.

ExternalProject has a somewhat similar flaw. I've tried once to use it for
FFmpeg and found out that ExternalProject does its work at the build step.
On the configure step there is nothing yet and consequently there is no way
to figure out what will be needed to link against the library. Even though
FFmpeg does report its dependencies (via pkg-config) later after being
installed, it is too late. There was a workaround with two-step build but
that looked ugly, scaled even worse and I've settled on separate builds and
custom find modules.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Raymond Wan]

Hi,


On Tue, Jun 28, 2016 at 8:41 PM, Cfyz  wrote:
> dependencies' locations. Personally I do not agree with that as I see
> neither any way around specifying paths to dependencies not any substantial
> harm in that. Maybe I am missing something but as far as I've found there
> was not discussion about this matter before.


Actually, there has been discussions about transitive dependencies
before.  In fact, such discussions crop up once a month or two.

See for example,
http://public.kitware.com/pipermail/cmake/2016-January/062590.html
(which I did reply, also with a link to my GitHub repository with a
minimal example).  Perhaps many others before me have also done the
same.  [Again, just because I reply doesn't mean I think I'm
right...I'm learning through discussions, as well!]

In my humble opinion, Sven's problem is two-fold.  They are
inter-related, but they could be separated.  The first is the
transitive dependency and for this, if it's all within the same source
code tree then something like what I did could work.  The second is
that the dependencies are not installed in system-level directories,
even though multiple users on a super-computer need to share them.

Because of this second factor, I don't have an idea what Sven should
do.  Indeed, what you suggest could work -- as I don't have a need, I
haven't given that much thought.

I think if I were presented with the same problem as Sven, I'd ask the
system administrator to install it.  Or, if that wasn't possible, use
ExternalProject.  Yes, if there are 100 users, then there would be 100
copies of the dependency.  But, running CMake will grab the most
recent copy from the source repository.  It's inefficient in terms of
disk space, but at least A (assuming A depends on B) will be built
correctly.  I don't know if CMake's aim is to build software correctly
*and* save disk space...  :-)

Ray
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey Cfyz,

After reading your post to the list, it does indeed seem like we have
the same problem. I also wrote a lot of homebrew code you mentioned at
the end of the post, but it indeed always seems to always break
something for some user of the code, even though it seems to work for
everyone else. Hence my question if there is some "right" way of doing
this with CMake, without all of the code I wrote to work around the problem.

In the mean time I'll have a look at the code you wrote, but it seems
very similar to something I already had, which broke for some users.

Sven

On 06/28/2016 02:35 PM, Cfyz wrote:
> Hello.
>
> I've posted a message a few days ago on a very similar subject. Didn't
> know there was a discussion already.
>
> The main problem in my opinion is that CMake target exporting system
> does not handle transitive dependencies at all. CMake handles them
> correctly at the build step, but not at the export. As I've noted in
> my message there is a small hint on that being a conscious decision:
> documentation about target_link_libraries and relevant target
> properties advises not to include paths to dependencies into the
> target interface as it would hard-code dependencies' locations.
> Personally I do not agree with that as I see neither any way around
> specifying paths to dependencies not any substantial harm in that.
> Maybe I am missing something but as far as I've found there was not
> discussion about this matter before.
>
> With that there are some ways to make transitive dependencies work.
> Though you'll have to forsake most of CMake's export machinery.
> Probably the easiest way is to make each library export it's full
> interface in the Foo_INCLUDE_DIRS / Foo_LIBRARIES variables. By
> writing the AConfig.cmake by hand you can figure out and export the
> full path to the A. Therefore AConfig.cmake will essentially be:
>
>   get_filename_component(A_CONFIG_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
>   find_library(A_LIB "A" HINTS ${A_CONFIG_DIR}/../../../lib)
>   set(A_INCLUDE_DIRS ${A_CONFIG_DIR}/../../../include)
>   set(A_LIBRARIES ${A_LIB})
>   set(A_FOUND TRUE)
>
> From here on, every other library which care about its dependencies
> will have to export its interface in a similar way. However, since
> BConfig.cmake cannot find A by itself, it will need some help from the
> project configuration step. B project will need to gather its
> interface and do a configure_file() fixing B's interface dependencies
> in it:
>
>   find_package(A)
>   list(APPEND B_INTERFACE_INCLUDE_DIRS ${A_INCLUDE_DIRS})
>   list(APPEND B_INTERFACE_LIBRARIES ${A_LIBRARIES}) # here comes the
> full path to A library
>   ...
>   configure_file(BConfig.cmake.in 
> ${CMAKE_CURRENT_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/BConfig.cmake @ONLY)
>
> Where BConfig.cmake.in  may look like:
>
>   get_filename_component(B_CONFIG_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
>   find_library(B_LIB "B" HINTS ${B_CONFIG_DIR}/../../../lib)
>   set(B_INCLUDE_DIRS ${B_CONFIG_DIR}/../../../include
> @B_INTERFACE_INCLUDE_DIRS@)
>   set(B_LIBRARIES ${B_LIB} @B_INTERFACE_LIBRARIES@)
>   set(B_FOUND TRUE)
>
> Now find_package(B) will export B_INCLUDE_DIRS / B_LIBRARIES which
> will list full paths to everything necessary to build against it.
>
> At my company we usually use static libraries so that it's rarely a
> problem but with dynamic libraries (.so) this approach will not be
> enough to run the app. Searching for dynamic libraries is not exactly
> the build system responsibility, though and greatly depends on the
> platform and the way you distribute the resulting application. If
> anything it always possible to make another transitive Foo_RPATHS
> property.
>
> This 'simple' approach does not work with libraries that export
> 'imported targets' only (if I recall correctly, the Qt find module
> exports Qt::XYZ targets but not Qt_LIBRARIES) so that there is no
> interface variables to merge. In this case one would need to iterate
> over imported targets and merge their INTERFACE_xxx properties in a
> similar way.
>
> I believe this to be an oversight. Everything should be fine if there
> was an option to resolve and export transitive dependencies along the
> targets in CMake export() command. Probably this may be fixed by an
> external helper script (similar to CMakePackageConfigHelpers module)
> providing some function that takes a list of targets, analyzes them
> and writes a config file complete with targets and their dependencies.
>
> On 28 June 2016 at 13:35, Sven Baars  > wrote:
>
> Hey Ray,
>
> Project A is used by many projects, and so is project B. So
> ExternalProject is not the right solution. I just want to be able
> to fix
> project B in such a way that people, including myself, can easily
> use it
> without knowledge of where project A is located. The packages will all
> be used mostly on 

Re: [CMake] How do you handle recursive dependencies in CMake

[Cfyz]

Hello.

I've posted a message a few days ago on a very similar subject. Didn't know
there was a discussion already.

The main problem in my opinion is that CMake target exporting system does
not handle transitive dependencies at all. CMake handles them correctly at
the build step, but not at the export. As I've noted in my message there is
a small hint on that being a conscious decision: documentation about
target_link_libraries and relevant target properties advises not to include
paths to dependencies into the target interface as it would hard-code
dependencies' locations. Personally I do not agree with that as I see
neither any way around specifying paths to dependencies not any substantial
harm in that. Maybe I am missing something but as far as I've found there
was not discussion about this matter before.

With that there are some ways to make transitive dependencies work. Though
you'll have to forsake most of CMake's export machinery. Probably the
easiest way is to make each library export it's full interface in the
Foo_INCLUDE_DIRS / Foo_LIBRARIES variables. By writing the AConfig.cmake by
hand you can figure out and export the full path to the A. Therefore
AConfig.cmake will essentially be:

  get_filename_component(A_CONFIG_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
  find_library(A_LIB "A" HINTS ${A_CONFIG_DIR}/../../../lib)
  set(A_INCLUDE_DIRS ${A_CONFIG_DIR}/../../../include)
  set(A_LIBRARIES ${A_LIB})
  set(A_FOUND TRUE)

>From here on, every other library which care about its dependencies will
have to export its interface in a similar way. However, since BConfig.cmake
cannot find A by itself, it will need some help from the project
configuration step. B project will need to gather its interface and do a
configure_file() fixing B's interface dependencies in it:

  find_package(A)
  list(APPEND B_INTERFACE_INCLUDE_DIRS ${A_INCLUDE_DIRS})
  list(APPEND B_INTERFACE_LIBRARIES ${A_LIBRARIES}) # here comes the full
path to A library
  ...
  configure_file(BConfig.cmake.in
${CMAKE_CURRENT_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/BConfig.cmake @ONLY)

Where BConfig.cmake.in may look like:

  get_filename_component(B_CONFIG_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
  find_library(B_LIB "B" HINTS ${B_CONFIG_DIR}/../../../lib)
  set(B_INCLUDE_DIRS ${B_CONFIG_DIR}/../../../include
@B_INTERFACE_INCLUDE_DIRS@)
  set(B_LIBRARIES ${B_LIB} @B_INTERFACE_LIBRARIES@)
  set(B_FOUND TRUE)

Now find_package(B) will export B_INCLUDE_DIRS / B_LIBRARIES which will
list full paths to everything necessary to build against it.

At my company we usually use static libraries so that it's rarely a problem
but with dynamic libraries (.so) this approach will not be enough to run
the app. Searching for dynamic libraries is not exactly the build system
responsibility, though and greatly depends on the platform and the way you
distribute the resulting application. If anything it always possible to
make another transitive Foo_RPATHS property.

This 'simple' approach does not work with libraries that export 'imported
targets' only (if I recall correctly, the Qt find module exports Qt::XYZ
targets but not Qt_LIBRARIES) so that there is no interface variables to
merge. In this case one would need to iterate over imported targets and
merge their INTERFACE_xxx properties in a similar way.

I believe this to be an oversight. Everything should be fine if there was
an option to resolve and export transitive dependencies along the targets
in CMake export() command. Probably this may be fixed by an external helper
script (similar to CMakePackageConfigHelpers module) providing some
function that takes a list of targets, analyzes them and writes a config
file complete with targets and their dependencies.

On 28 June 2016 at 13:35, Sven Baars  wrote:

> Hey Ray,
>
> Project A is used by many projects, and so is project B. So
> ExternalProject is not the right solution. I just want to be able to fix
> project B in such a way that people, including myself, can easily use it
> without knowledge of where project A is located. The packages will all
> be used mostly on supercomputers, where users never have rights to
> install anything, and even if they install them as a module, it is still
> in a non-standard path. As you can see from the example, this works fine
> for project B which depends on A, but not for project C which depends on
> B. Even if you set the PATH correctly.
>
> Sven
>
> On 06/28/2016 12:25 PM, Raymond Wan wrote:
> > Hi Sven,
> >
> >
> > On Tue, Jun 28, 2016 at 6:03 PM, Sven Baars  wrote:
> >> The packages I use are installed in a non-standard path, because I don't
> >> have access to the system directories on most systems I work on, but are
> >> used by many other libraries. They are also all separate packages, not
> >> packages in the same source tree. I did not see my attachment on the
> >> mailing list archive, so instead I just put it on Github. It can be
> >> found here:
> >>
> >> 

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey Ray,

Project A is used by many projects, and so is project B. So
ExternalProject is not the right solution. I just want to be able to fix
project B in such a way that people, including myself, can easily use it
without knowledge of where project A is located. The packages will all
be used mostly on supercomputers, where users never have rights to
install anything, and even if they install them as a module, it is still
in a non-standard path. As you can see from the example, this works fine
for project B which depends on A, but not for project C which depends on
B. Even if you set the PATH correctly.

Sven

On 06/28/2016 12:25 PM, Raymond Wan wrote:
> Hi Sven,
>
>
> On Tue, Jun 28, 2016 at 6:03 PM, Sven Baars  wrote:
>> The packages I use are installed in a non-standard path, because I don't
>> have access to the system directories on most systems I work on, but are
>> used by many other libraries. They are also all separate packages, not
>> packages in the same source tree. I did not see my attachment on the
>> mailing list archive, so instead I just put it on Github. It can be
>> found here:
>>
>> https://github.com/Sbte/cmake-example
>>
>> I hope the example shows my workflow, and also what doesn't work for me.
>
> H, in that case, I should probably remain quiet as this is not
> quite something I've had to do.
>
> Maybe the only advice I can give is, if you plan to distribute your
> work to others, what are your expectations in terms of where do you
> think users should install these dependencies.
>
> For example, if it's a system-level directory, then maybe you can
> consider a FIND_PACKAGE solution which has a list of default paths to
> search.  When you're developing on your own computer, you just change
> the default path to include paths in your home directory.
>
> There is also the ExternalProject () directive which you could
> consider [https://cmake.org/cmake/help/v3.3/module/ExternalProject.html]
> .  This would retrieve the dependencies from outside sources (i.e.,
> repositories) and build them.  This would keep everything within the
> same build/ path and I haven't done that (and didn't mention it
> before) because all of my packages are within the same source tree.
>
> So, still not quite your situation, though.
>
> I hope this helps!  Perhaps others can chime in and help you with
> exactly what you're stuck with...
>
> Ray

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Raymond Wan]

Hi Sven,


On Tue, Jun 28, 2016 at 6:03 PM, Sven Baars  wrote:
> The packages I use are installed in a non-standard path, because I don't
> have access to the system directories on most systems I work on, but are
> used by many other libraries. They are also all separate packages, not
> packages in the same source tree. I did not see my attachment on the
> mailing list archive, so instead I just put it on Github. It can be
> found here:
>
> https://github.com/Sbte/cmake-example
>
> I hope the example shows my workflow, and also what doesn't work for me.


H, in that case, I should probably remain quiet as this is not
quite something I've had to do.

Maybe the only advice I can give is, if you plan to distribute your
work to others, what are your expectations in terms of where do you
think users should install these dependencies.

For example, if it's a system-level directory, then maybe you can
consider a FIND_PACKAGE solution which has a list of default paths to
search.  When you're developing on your own computer, you just change
the default path to include paths in your home directory.

There is also the ExternalProject () directive which you could
consider [https://cmake.org/cmake/help/v3.3/module/ExternalProject.html]
.  This would retrieve the dependencies from outside sources (i.e.,
repositories) and build them.  This would keep everything within the
same build/ path and I haven't done that (and didn't mention it
before) because all of my packages are within the same source tree.

So, still not quite your situation, though.

I hope this helps!  Perhaps others can chime in and help you with
exactly what you're stuck with...

Ray
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hi Ray,

The packages I use are installed in a non-standard path, because I don't
have access to the system directories on most systems I work on, but are
used by many other libraries. They are also all separate packages, not
packages in the same source tree. I did not see my attachment on the
mailing list archive, so instead I just put it on Github. It can be
found here:

https://github.com/Sbte/cmake-example

I hope the example shows my workflow, and also what doesn't work for me.

Sven

On 06/28/2016 11:52 AM, Raymond Wan wrote:
> Hi Sven,
>
>
> On Tue, Jun 28, 2016 at 5:26 PM, Sven Baars  wrote:
>> Hey all,
>>
>> Since I did not receive a reply to my previous mail, I decided to write a
>> minimal example explaining my problem, where I kept al the names as I used
>> in my previous mails. I hope that after seeing my example, someone can
>> explain me what I am doing wrong here. I included a script that reproduces
>> the problem.
>>
>> Cheers,
>> Sven
>
> I'm not much of an expert with CMake, but I can explain to you what
> I've done, at least.
>
> If I understand correctly, package X makes use of package Y, but
> during compilation, it cannot find it?  I think what you can do
> depends on whether X and Y are in two separate source trees.
>
> If they are, then it is almost like Y is the zlib library.  Then what
> you can do is, like the zlib library, install it into a system
> directory (i.e., via INSTALL ()) and then X will use a FIND_PACKAGE to
> look for it.
>
> But, suppose Y isn't like zlib.  For example, it is a library that is
> within the same source tree and will only ever be used by X (both of
> which are written by you).  I think this is the scenario that you're
> talking about?  In my case, I have done this in my CMakeLists.txt:
>
> SET (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
>
> I guess you can set it
> [https://cmake.org/cmake/help/v3.3/variable/CMAKE_LIBRARY_OUTPUT_DIRECTORY.html]
> to anything so that all of the libraries will be installed into one
> place without doing an INSTALL.  Of course, this directory is or
> somewhere within the build/ directory.  (I suppose doing an install
> into ${CMAKE_BINARY_DIR} would achieve the same thing?)
>
> I had previously installed Y's archive into a system directory, but
> besides the need to run FIND_PACKAGE, I thought it was "silly" to
> install an archive into a system directory that is only used by my own
> program.  Seems a bit messy and intrusive to the end-user -- my
> libraries are quite unlike zlib; few chance someone else will use it
> :-) .  At the very least, this approach keeps it in one place within
> build/ .
>
> I'm not sure if it is the "right" way to do it, but this is what I do
> at the moment...  I hope it helps!
>
> Ray

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Raymond Wan]

Hi Sven,


On Tue, Jun 28, 2016 at 5:26 PM, Sven Baars  wrote:
> Hey all,
>
> Since I did not receive a reply to my previous mail, I decided to write a
> minimal example explaining my problem, where I kept al the names as I used
> in my previous mails. I hope that after seeing my example, someone can
> explain me what I am doing wrong here. I included a script that reproduces
> the problem.
>
> Cheers,
> Sven


I'm not much of an expert with CMake, but I can explain to you what
I've done, at least.

If I understand correctly, package X makes use of package Y, but
during compilation, it cannot find it?  I think what you can do
depends on whether X and Y are in two separate source trees.

If they are, then it is almost like Y is the zlib library.  Then what
you can do is, like the zlib library, install it into a system
directory (i.e., via INSTALL ()) and then X will use a FIND_PACKAGE to
look for it.

But, suppose Y isn't like zlib.  For example, it is a library that is
within the same source tree and will only ever be used by X (both of
which are written by you).  I think this is the scenario that you're
talking about?  In my case, I have done this in my CMakeLists.txt:

SET (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})

I guess you can set it
[https://cmake.org/cmake/help/v3.3/variable/CMAKE_LIBRARY_OUTPUT_DIRECTORY.html]
to anything so that all of the libraries will be installed into one
place without doing an INSTALL.  Of course, this directory is or
somewhere within the build/ directory.  (I suppose doing an install
into ${CMAKE_BINARY_DIR} would achieve the same thing?)

I had previously installed Y's archive into a system directory, but
besides the need to run FIND_PACKAGE, I thought it was "silly" to
install an archive into a system directory that is only used by my own
program.  Seems a bit messy and intrusive to the end-user -- my
libraries are quite unlike zlib; few chance someone else will use it
:-) .  At the very least, this approach keeps it in one place within
build/ .

I'm not sure if it is the "right" way to do it, but this is what I do
at the moment...  I hope it helps!

Ray
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey all,

Since I did not receive a reply to my previous mail, I decided to write
a minimal example explaining my problem, where I kept al the names as I
used in my previous mails. I hope that after seeing my example, someone
can explain me what I am doing wrong here. I included a script that
reproduces the problem.

Cheers,
Sven

On 06/10/2016 01:34 PM, Sven Baars wrote:
> Hey Chuck,
>
> In ATargets.cmake it says
>
> add_library(a SHARED IMPORTED)
>
> and in BTargets.cmake it says
>
> add_library(b SHARED IMPORTED)
>
> However, if I build C it will say
>
> /usr/bin/ld: cannot find -la
>
> when I do
>
> find_package(B)
> target_link_libraries(c b)
> add_executable(main main.cpp)
> target_link_libraries(main c)
>
> This, I think, because the 'b' target depends on 'a' but doesn't know
> where 'a' is located, because this is not described in BTargets.cmake.
>
> Sven
>
> On 06/08/2016 06:07 PM, Chuck Atkins wrote:
>> If the projects are each providing their own Config.cmake file then
>> you probably will want to use those instead of Find modules.  Instead
>> of using the Foo_Bar_LIBRARIES variable though, try using the actual
>> imported targets from the config files.  Those should have the
>> appropriate dependency information. If you're not sure what they are
>> you should be able to dig through the provided Config.cmake and
>> Targets.cmake files for FooBar and see which targets are created bu
>> "add_library(FooBar::Foo ... IMPORTED)".
>>
>> - Chuck
>>
>> On Wed, Jun 8, 2016 at 11:31 AM, Sven Baars > > wrote:
>>
>> In my case all projects provide their own FooBarConfig.cmake, but
>> not a FindFooBar.cmake. For this reason I wanted to use those,
>> because I thought it saves me the effort of writing a lot of
>> FindFooBar.cmake files, and it also seemed like the right way to
>> do it, because it provides things like FooBar_LIBRARIES, which is
>> what I need. The reason I said that I create my own
>> FooBarConfig.cmake, is because I want to use CMake properly in my
>> own project, so I also want to provide a FoorBarConfig.cmake file
>> for my users. So even if the projects I use don't do everything
>> properly, I still want to do it properly myself. The problem in
>> this case is that I'm not sure what the proper way is. If the
>> FoorBarConfig.cmake file is not enough to handle the dependencies
>> I encounter, should I provide a FindFooBar.cmake file or
>> something, instead of just the FooBarConfig.cmake? And meanwhile
>> also write a FindFooBar.cmake file for all my dependencies?
>>
>> Another reason why this seems odd to me, is because if N
>> different projects use FooBar, then also possibly N different
>> versions of FindFooBar.cmake are created by all the different
>> projects (I have actually already seen this a lot). That is the
>> case, because the FindFooBar.cmake file is not provided by the
>> FooBar project, unlike the FooBarConfig.cmake.
>>
>> Cheers,
>> Sven
>>
>>
>> On 06/08/2016 03:11 PM, Chuck Atkins wrote:
>>> The FooBarConfig.cmake is something that should be generated by
>>> FooBar's build.   The reason you don't get absolute paths for
>>> the "libraries" from a package config file is that they're not
>>> actually libraries but imported targets.  The imported target
>>> let's you treat "foo" as though it were a library built by your
>>> project.  It then has the appropriate target properties set on
>>> it ti define the full path to it's library, etc.  That being
>>> said, if you're manually creating the FooBarConfig.cmake that's
>>> not really the right approach.  If the FooBar buil;d doesn't
>>> actually generate it's own FooBarConfig.cmake file then you'll
>>> want to create you're own FindFooBar.cmake.  A bare bones find
>>> module that creates an imported target would look something like
>>> this:
>>>
>>> if(NOT FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>>>   find_path(FooBar_INCLUDE_DIR FooBar.h)
>>>   find_library(FooBar_LIBRARY FooBar)
>>>
>>>   include(FindPackageHandleStandardArgs)
>>>   find_package_handle_standard_args(FooBar
>>> FOUND_VAR FooBar_FOUND
>>> REQUIRED_VARS FooBar_INCLUDE_DIR FooBar_LIBRARY
>>>   )
>>> endif()
>>>
>>> if(FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>>>   add_library(FooBar::FooBar UNKNOWN IMPORTED)
>>>   set_target_properties(FooBar::FooBar PROPERTIES
>>> IMPORTED_LOCATION ${FooBar_LIBRARY}
>>> INTERFACE_INCLUDE_DIRECTORIES ${FooBar_INCLUDE_DIR}
>>>   )
>>> endif()
>>>
>>> Then in your project you can use:
>>>
>>> find_package(FooBar)
>>> add_library(MyLib supercoolfiles.cxx)
>>> target_libkLibraries(MyLib FooBar::FooBar)
>>>
>>> Where this starts to get really helpful in your situation is if
>>> you have an imported target for A, B, and C, 

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

Hey Chuck,

In ATargets.cmake it says

add_library(a SHARED IMPORTED)

and in BTargets.cmake it says

add_library(b SHARED IMPORTED)

However, if I build C it will say

/usr/bin/ld: cannot find -la

when I do

find_package(B)
target_link_libraries(c b)
add_executable(main main.cpp)
target_link_libraries(main c)

This, I think, because the 'b' target depends on 'a' but doesn't know
where 'a' is located, because this is not described in BTargets.cmake.

Sven

On 06/08/2016 06:07 PM, Chuck Atkins wrote:
> If the projects are each providing their own Config.cmake file then
> you probably will want to use those instead of Find modules.  Instead
> of using the Foo_Bar_LIBRARIES variable though, try using the actual
> imported targets from the config files.  Those should have the
> appropriate dependency information. If you're not sure what they are
> you should be able to dig through the provided Config.cmake and
> Targets.cmake files for FooBar and see which targets are created bu
> "add_library(FooBar::Foo ... IMPORTED)".
>
> - Chuck
>
> On Wed, Jun 8, 2016 at 11:31 AM, Sven Baars  > wrote:
>
> In my case all projects provide their own FooBarConfig.cmake, but
> not a FindFooBar.cmake. For this reason I wanted to use those,
> because I thought it saves me the effort of writing a lot of
> FindFooBar.cmake files, and it also seemed like the right way to
> do it, because it provides things like FooBar_LIBRARIES, which is
> what I need. The reason I said that I create my own
> FooBarConfig.cmake, is because I want to use CMake properly in my
> own project, so I also want to provide a FoorBarConfig.cmake file
> for my users. So even if the projects I use don't do everything
> properly, I still want to do it properly myself. The problem in
> this case is that I'm not sure what the proper way is. If the
> FoorBarConfig.cmake file is not enough to handle the dependencies
> I encounter, should I provide a FindFooBar.cmake file or
> something, instead of just the FooBarConfig.cmake? And meanwhile
> also write a FindFooBar.cmake file for all my dependencies?
>
> Another reason why this seems odd to me, is because if N different
> projects use FooBar, then also possibly N different versions of
> FindFooBar.cmake are created by all the different projects (I have
> actually already seen this a lot). That is the case, because the
> FindFooBar.cmake file is not provided by the FooBar project,
> unlike the FooBarConfig.cmake.
>
> Cheers,
> Sven
>
>
> On 06/08/2016 03:11 PM, Chuck Atkins wrote:
>> The FooBarConfig.cmake is something that should be generated by
>> FooBar's build.   The reason you don't get absolute paths for the
>> "libraries" from a package config file is that they're not
>> actually libraries but imported targets.  The imported target
>> let's you treat "foo" as though it were a library built by your
>> project.  It then has the appropriate target properties set on it
>> ti define the full path to it's library, etc.  That being said,
>> if you're manually creating the FooBarConfig.cmake that's not
>> really the right approach.  If the FooBar buil;d doesn't actually
>> generate it's own FooBarConfig.cmake file then you'll want to
>> create you're own FindFooBar.cmake.  A bare bones find module
>> that creates an imported target would look something like this:
>>
>> if(NOT FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>>   find_path(FooBar_INCLUDE_DIR FooBar.h)
>>   find_library(FooBar_LIBRARY FooBar)
>>
>>   include(FindPackageHandleStandardArgs)
>>   find_package_handle_standard_args(FooBar
>> FOUND_VAR FooBar_FOUND
>> REQUIRED_VARS FooBar_INCLUDE_DIR FooBar_LIBRARY
>>   )
>> endif()
>>
>> if(FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>>   add_library(FooBar::FooBar UNKNOWN IMPORTED)
>>   set_target_properties(FooBar::FooBar PROPERTIES
>> IMPORTED_LOCATION ${FooBar_LIBRARY}
>> INTERFACE_INCLUDE_DIRECTORIES ${FooBar_INCLUDE_DIR}
>>   )
>> endif()
>>
>> Then in your project you can use:
>>
>> find_package(FooBar)
>> add_library(MyLib supercoolfiles.cxx)
>> target_libkLibraries(MyLib FooBar::FooBar)
>>
>> Where this starts to get really helpful in your situation is if
>> you have an imported target for A, B, and C, then you can create
>> the appropriate dependencies.  Say, for example, you have a
>> FindA.cmake that follows the pattern above and generates an A::A
>> target.  Then in your FindB.cmake you can have:
>>
>> if(NOT B_FOUND AND NOT TARGET B::B)
>>   find_path(B_INCLUDE_DIR B.h)
>>   find_library(B_LIBRARY B)
>>
>>   include(FindPackageHandleStandardArgs)
>>   find_package_handle_standard_args(B
>> FOUND_VAR B_FOUND
>> REQUIRED_VARS B_INCLUDE_DIR B_LIBRARY
>>  

Re: [CMake] How do you handle recursive dependencies in CMake

[Chuck Atkins]

If the projects are each providing their own Config.cmake file then you
probably will want to use those instead of Find modules.  Instead of using
the Foo_Bar_LIBRARIES variable though, try using the actual imported
targets from the config files.  Those should have the appropriate
dependency information. If you're not sure what they are you should be able
to dig through the provided Config.cmake and Targets.cmake files for FooBar
and see which targets are created bu "add_library(FooBar::Foo ...
IMPORTED)".

- Chuck

On Wed, Jun 8, 2016 at 11:31 AM, Sven Baars  wrote:

> In my case all projects provide their own FooBarConfig.cmake, but not a
> FindFooBar.cmake. For this reason I wanted to use those, because I thought
> it saves me the effort of writing a lot of FindFooBar.cmake files, and it
> also seemed like the right way to do it, because it provides things like
> FooBar_LIBRARIES, which is what I need. The reason I said that I create my
> own FooBarConfig.cmake, is because I want to use CMake properly in my own
> project, so I also want to provide a FoorBarConfig.cmake file for my users.
> So even if the projects I use don't do everything properly, I still want to
> do it properly myself. The problem in this case is that I'm not sure what
> the proper way is. If the FoorBarConfig.cmake file is not enough to handle
> the dependencies I encounter, should I provide a FindFooBar.cmake file or
> something, instead of just the FooBarConfig.cmake? And meanwhile also write
> a FindFooBar.cmake file for all my dependencies?
>
> Another reason why this seems odd to me, is because if N different
> projects use FooBar, then also possibly N different versions of
> FindFooBar.cmake are created by all the different projects (I have actually
> already seen this a lot). That is the case, because the FindFooBar.cmake
> file is not provided by the FooBar project, unlike the FooBarConfig.cmake.
>
> Cheers,
> Sven
>
>
> On 06/08/2016 03:11 PM, Chuck Atkins wrote:
>
> The FooBarConfig.cmake is something that should be generated by FooBar's
> build.   The reason you don't get absolute paths for the "libraries" from a
> package config file is that they're not actually libraries but imported
> targets.  The imported target let's you treat "foo" as though it were a
> library built by your project.  It then has the appropriate target
> properties set on it ti define the full path to it's library, etc.  That
> being said, if you're manually creating the FooBarConfig.cmake that's not
> really the right approach.  If the FooBar buil;d doesn't actually generate
> it's own FooBarConfig.cmake file then you'll want to create you're own
> FindFooBar.cmake.  A bare bones find module that creates an imported target
> would look something like this:
>
> if(NOT FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>   find_path(FooBar_INCLUDE_DIR FooBar.h)
>   find_library(FooBar_LIBRARY FooBar)
>
>   include(FindPackageHandleStandardArgs)
>   find_package_handle_standard_args(FooBar
> FOUND_VAR FooBar_FOUND
> REQUIRED_VARS FooBar_INCLUDE_DIR FooBar_LIBRARY
>   )
> endif()
>
> if(FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>   add_library(FooBar::FooBar UNKNOWN IMPORTED)
>   set_target_properties(FooBar::FooBar PROPERTIES
> IMPORTED_LOCATION ${FooBar_LIBRARY}
> INTERFACE_INCLUDE_DIRECTORIES ${FooBar_INCLUDE_DIR}
>   )
> endif()
>
> Then in your project you can use:
>
> find_package(FooBar)
> add_library(MyLib supercoolfiles.cxx)
> target_libkLibraries(MyLib FooBar::FooBar)
>
> Where this starts to get really helpful in your situation is if you have
> an imported target for A, B, and C, then you can create the appropriate
> dependencies.  Say, for example, you have a FindA.cmake that follows the
> pattern above and generates an A::A target.  Then in your FindB.cmake you
> can have:
>
> if(NOT B_FOUND AND NOT TARGET B::B)
>   find_path(B_INCLUDE_DIR B.h)
>   find_library(B_LIBRARY B)
>
>   include(FindPackageHandleStandardArgs)
>   find_package_handle_standard_args(B
> FOUND_VAR B_FOUND
> REQUIRED_VARS B_INCLUDE_DIR B_LIBRARY
>   )
>
>
>
>
> *  find_package(A QUIET)   if(A_FOUND) set(B_Extra_Deps A::A)
> endif()*
> endif()
>
> if(B_FOUND AND NOT TARGET B::B)
>   add_library(B::B UNKNOWN IMPORTED)
>   set_target_properties(B::B PROPERTIES
> IMPORTED_LOCATION ${B_LIBRARY}
> INTERFACE_INCLUDE_DIRECTORIES ${B_INCLUDE_DIR}
>   )
>
>
>
>
> *  if(B_Extra_Deps) set_target_properties(B::B PROPERTIES
> INTERFACE_LINK_LIBRARIES ${B_Extra_Deps} )   endif()*
> endif()
>
> and similarly for FindC.cmake.  Since A::A, B::B, and C::C are actual
> proper CMake targets and not just library files then they carry with them
> associated target dependency information.  In CMake vernacular we refer
> this as "target usage requirements" since the target caries with it all the
> necessary information for something to actually use it, whether that's just
> extra link libraries or also extra include directories and 

Re: [CMake] How do you handle recursive dependencies in CMake

[Sven Baars]

In my case all projects provide their own FooBarConfig.cmake, but not a
FindFooBar.cmake. For this reason I wanted to use those, because I
thought it saves me the effort of writing a lot of FindFooBar.cmake
files, and it also seemed like the right way to do it, because it
provides things like FooBar_LIBRARIES, which is what I need. The reason
I said that I create my own FooBarConfig.cmake, is because I want to use
CMake properly in my own project, so I also want to provide a
FoorBarConfig.cmake file for my users. So even if the projects I use
don't do everything properly, I still want to do it properly myself. The
problem in this case is that I'm not sure what the proper way is. If the
FoorBarConfig.cmake file is not enough to handle the dependencies I
encounter, should I provide a FindFooBar.cmake file or something,
instead of just the FooBarConfig.cmake? And meanwhile also write a
FindFooBar.cmake file for all my dependencies?

Another reason why this seems odd to me, is because if N different
projects use FooBar, then also possibly N different versions of
FindFooBar.cmake are created by all the different projects (I have
actually already seen this a lot). That is the case, because the
FindFooBar.cmake file is not provided by the FooBar project, unlike the
FooBarConfig.cmake.

Cheers,
Sven

On 06/08/2016 03:11 PM, Chuck Atkins wrote:
> The FooBarConfig.cmake is something that should be generated by
> FooBar's build.   The reason you don't get absolute paths for the
> "libraries" from a package config file is that they're not actually
> libraries but imported targets.  The imported target let's you treat
> "foo" as though it were a library built by your project.  It then has
> the appropriate target properties set on it ti define the full path to
> it's library, etc.  That being said, if you're manually creating the
> FooBarConfig.cmake that's not really the right approach.  If the
> FooBar buil;d doesn't actually generate it's own FooBarConfig.cmake
> file then you'll want to create you're own FindFooBar.cmake.  A bare
> bones find module that creates an imported target would look something
> like this:
>
> if(NOT FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>   find_path(FooBar_INCLUDE_DIR FooBar.h)
>   find_library(FooBar_LIBRARY FooBar)
>
>   include(FindPackageHandleStandardArgs)
>   find_package_handle_standard_args(FooBar
> FOUND_VAR FooBar_FOUND
> REQUIRED_VARS FooBar_INCLUDE_DIR FooBar_LIBRARY
>   )
> endif()
>
> if(FooBar_FOUND AND NOT TARGET FooBar::FooBar)
>   add_library(FooBar::FooBar UNKNOWN IMPORTED)
>   set_target_properties(FooBar::FooBar PROPERTIES
> IMPORTED_LOCATION ${FooBar_LIBRARY}
> INTERFACE_INCLUDE_DIRECTORIES ${FooBar_INCLUDE_DIR}
>   )
> endif()
>
> Then in your project you can use:
>
> find_package(FooBar)
> add_library(MyLib supercoolfiles.cxx)
> target_libkLibraries(MyLib FooBar::FooBar)
>
> Where this starts to get really helpful in your situation is if you
> have an imported target for A, B, and C, then you can create the
> appropriate dependencies.  Say, for example, you have a FindA.cmake
> that follows the pattern above and generates an A::A target.  Then in
> your FindB.cmake you can have:
>
> if(NOT B_FOUND AND NOT TARGET B::B)
>   find_path(B_INCLUDE_DIR B.h)
>   find_library(B_LIBRARY B)
>
>   include(FindPackageHandleStandardArgs)
>   find_package_handle_standard_args(B
> FOUND_VAR B_FOUND
> REQUIRED_VARS B_INCLUDE_DIR B_LIBRARY
>   )
>
> *  find_package(A QUIET)
>   if(A_FOUND)
> set(B_Extra_Deps A::A)
>   endif()*
> endif()
>
> if(B_FOUND AND NOT TARGET B::B)
>   add_library(B::B UNKNOWN IMPORTED)
>   set_target_properties(B::B PROPERTIES
> IMPORTED_LOCATION ${B_LIBRARY}
> INTERFACE_INCLUDE_DIRECTORIES ${B_INCLUDE_DIR}
>   )
> *  if(B_Extra_Deps)
> set_target_properties(B::B PROPERTIES
>   INTERFACE_LINK_LIBRARIES ${B_Extra_Deps}
> )
>   endif()*
> endif()
>
> and similarly for FindC.cmake.  Since A::A, B::B, and C::C are actual
> proper CMake targets and not just library files then they carry with
> them associated target dependency information.  In CMake vernacular we
> refer this as "target usage requirements" since the target caries with
> it all the necessary information for something to actually use it,
> whether that's just extra link libraries or also extra include
> directories and compile definitions needed.  The Package::Target
> naming convention is the idea that each package has it's own namespace
> where you may define multiple targets.

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects 

Re: [CMake] How do you handle recursive dependencies in CMake

[Chuck Atkins]

The FooBarConfig.cmake is something that should be generated by FooBar's
build.   The reason you don't get absolute paths for the "libraries" from a
package config file is that they're not actually libraries but imported
targets.  The imported target let's you treat "foo" as though it were a
library built by your project.  It then has the appropriate target
properties set on it ti define the full path to it's library, etc.  That
being said, if you're manually creating the FooBarConfig.cmake that's not
really the right approach.  If the FooBar buil;d doesn't actually generate
it's own FooBarConfig.cmake file then you'll want to create you're own
FindFooBar.cmake.  A bare bones find module that creates an imported target
would look something like this:

if(NOT FooBar_FOUND AND NOT TARGET FooBar::FooBar)
  find_path(FooBar_INCLUDE_DIR FooBar.h)
  find_library(FooBar_LIBRARY FooBar)

  include(FindPackageHandleStandardArgs)
  find_package_handle_standard_args(FooBar
FOUND_VAR FooBar_FOUND
REQUIRED_VARS FooBar_INCLUDE_DIR FooBar_LIBRARY
  )
endif()

if(FooBar_FOUND AND NOT TARGET FooBar::FooBar)
  add_library(FooBar::FooBar UNKNOWN IMPORTED)
  set_target_properties(FooBar::FooBar PROPERTIES
IMPORTED_LOCATION ${FooBar_LIBRARY}
INTERFACE_INCLUDE_DIRECTORIES ${FooBar_INCLUDE_DIR}
  )
endif()

Then in your project you can use:

find_package(FooBar)
add_library(MyLib supercoolfiles.cxx)
target_libkLibraries(MyLib FooBar::FooBar)

Where this starts to get really helpful in your situation is if you have an
imported target for A, B, and C, then you can create the appropriate
dependencies.  Say, for example, you have a FindA.cmake that follows the
pattern above and generates an A::A target.  Then in your FindB.cmake you
can have:

if(NOT B_FOUND AND NOT TARGET B::B)
  find_path(B_INCLUDE_DIR B.h)
  find_library(B_LIBRARY B)

  include(FindPackageHandleStandardArgs)
  find_package_handle_standard_args(B
FOUND_VAR B_FOUND
REQUIRED_VARS B_INCLUDE_DIR B_LIBRARY
  )




*  find_package(A QUIET)  if(A_FOUND)set(B_Extra_Deps A::A)  endif()*
endif()

if(B_FOUND AND NOT TARGET B::B)
  add_library(B::B UNKNOWN IMPORTED)
  set_target_properties(B::B PROPERTIES
IMPORTED_LOCATION ${B_LIBRARY}
INTERFACE_INCLUDE_DIRECTORIES ${B_INCLUDE_DIR}
  )




*  if(B_Extra_Deps)set_target_properties(B::B PROPERTIES
INTERFACE_LINK_LIBRARIES ${B_Extra_Deps})  endif()*
endif()

and similarly for FindC.cmake.  Since A::A, B::B, and C::C are actual
proper CMake targets and not just library files then they carry with them
associated target dependency information.  In CMake vernacular we refer
this as "target usage requirements" since the target caries with it all the
necessary information for something to actually use it, whether that's just
extra link libraries or also extra include directories and compile
definitions needed.  The Package::Target naming convention is the idea that
each package has it's own namespace where you may define multiple targets.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake