Re: [CMake] Interest in adding to CMake the CMakeGet module to get dependencies

[P F via CMake]

> On Aug 4, 2017, at 9:18 PM, Craig Scott  wrote:
> 
> TLDR: There may be other work already covering similar capabilities (e.g. 
> hunter and WIP that I'm uncloaking here). This response is mostly about 
> providing broader discussion.
> 
> 
> Paul it seems you've been busy since you first asked about cget 
>  on this list last 
> year. ;)

Thanks, cget has improved a lot in the last year.

> 
> In the interests of disclosure, this latest email of yours has prompted me to 
> uncloak some of my own work which may be relevant. This work has been in 
> development and testing on real world projects for the past 2 years or so. 
> Without wanting to hijack your thread, let me give a little info about it and 
> then see my responses to your comments at the end for how/why I think this 
> may be relevant to your request for comments.
> 
> ExternalProject can be used in a different way to allow it to perform 
> downloads at configure time rather than build time. You can see a relatively 
> straightforward implementation of this here:
> 
> https://github.com/Crascit/DownloadProject 
> 
> 

That seems to support the `add_subdirectory` workflow, but how would it work 
when using the `find_package` workflow? This workflow is useful to help package 
manager tools(including those which pulls binaries).

> This has the advantage of leveraging all of the various download methods 
> ExternalProject already provides, meaning they are already fairly widely used 
> and tested. Any improvements to ExternalProject also come for free and 
> documentation for the various methods is also provided (I recently overhauled 
> the ExternalProject docs and these will appear in the 3.10.0 release, but 
> they can be viewed in the master docs 
>  in the 
> meantime). ExternalProject also has the advantage that it supports updating 
> after the initial download in the case of repository sources like git, svn, 
> etc.
> 
> The uncloaking part of my response is that I have been working on a full 
> dependency download framework over the past year and half which is built on 
> top of the above DownloadProject implementation. It supports hierarchical 
> dependencies across complex project structures and makes it very easy for 
> projects to pull in other projects, including support for higher level 
> projects being able to override dependency details of projects lower in the 
> dependency tree if they want to. I've been incubating this privately in a 
> real world company environment to iron out the kinks and ensure the interface 
> that projects interact with supports the right set of features and that less 
> knowledgable users can understand it, etc. I'm planning on putting both 
> DownloadProject and the dependency framework up for review to merge into 
> CMake within the next few months, all going well. Whereas your work seems to 
> focus on building and/or re-using an installed/binary result, my work focuses 
> more on making external projects part of your main build. This has advantages 
> like using the same compiler settings, toolchain details, etc. and all of the 
> dependency targets are automatically available to the rest of the project if 
> the dependency uses CMake as its build system. I'll postpone further details 
> on it until it is ready for review, but that should be enough for context for 
> my responses to your comments further below.
> 
> There's also hunter , which you've already 
> been made aware of and which is closer to your proposal in terms of the set 
> of problems it tries to solve. Hunter has been around for a while, is 
> reasonably well known and on the face of it, already seems to do all the 
> things cmake-get is trying to do. It is also based on ExternalProject. It may 
> be helpful if you could show how your work differs from what hunter already 
> provides so that its value to the CMake community is clear.
> 
> Hopefully that's not too overwhelming for background/context. See my 
> responses in the remainder below.
> 
> 
> 
> On Sat, Aug 5, 2017 at 5:57 AM, paul via CMake  > wrote:
> Hi,
> 
> I have a written a cmake module to get dependencies using the cget protocol
> here:
> 
> https://github.com/pfultz2/cmake-get 
> 
> This is different than `ExternelProject`:
> 
> * `ExternelProject` happens only during build, which allows this module to
> work in both in config and script mode. 
> 
> As detailed above, you can use ExternalProject at configure time too and 
> there's already a relatively simple, generic implementation available showing 
> how to do it for the download case.
> 
>  
> * In config mode, the user can just do:
> 
> cmake_get( PREFIX ${CMAKE_CURRENT_BINARY_DIR}/deps)
> find_package( HINTS ${CMAKE_CURRENT_BINARY_DIR}/deps)

Re: [CMake] Interest in adding to CMake the CMakeGet module to get dependencies

[Veikko Eeva]

> TLDR: There may be other work already covering similar capabilities (e.g. 
> hunter and WIP that I'm uncloaking here). This response is mostly about 
> providing broader discussion.

https://github.com/Microsoft/vcpkg
https://github.com/conan-io/cmake-conan
https://github.com/Microsoft/vcpkg/issues/478


Cheers,
Veikko Eeva



Lähettäjä: CMake  käyttäjän Craig Scott 
 puolesta
Lähetetty: 5. elokuuta 2017 5:18
Vastaanottaja: paul
Kopio: cmake@cmake.org; Ruslan Baratov
Aihe: Re: [CMake] Interest in adding to CMake the CMakeGet module to get 
dependencies
  


TLDR: There may be other work already covering similar capabilities (e.g. 
hunter and WIP that I'm uncloaking here). This response is mostly about 
providing broader discussion.



 Paul it seems you've been busy since you first  asked about cget on this list 
last year. ;)


In the interests of disclosure, this latest email of yours has prompted me to 
uncloak some of my own work which may be relevant. This work has been in 
development and testing on real world projects for the past 2 years or so. 
Without wanting to hijack  your thread, let me give a little info about it and 
then see my responses to your comments at the end for how/why I think this may 
be relevant to your request for comments.


ExternalProject can be used in a different way to allow it to perform downloads 
at configure time rather than build time. You can see a relatively 
straightforward implementation of this here:


    https://github.com/Crascit/DownloadProject



This has the advantage of leveraging all of the various download methods 
ExternalProject already provides, meaning they are already fairly widely used 
and tested. Any improvements to ExternalProject also come for free and 
documentation for the various  methods is also provided (I recently overhauled 
the ExternalProject docs and these will appear in the 3.10.0 release, but they 
can be viewed in the master docs in the meantime). ExternalProject also has the 
advantage that it supports updating after the initial download in the case of 
repository sources like git, svn, etc.


The uncloaking part of my response is that I have been working on a full 
dependency download framework over the past year and half which is built on top 
of the above DownloadProject implementation. It supports hierarchical 
dependencies across complex project  structures and makes it very easy for 
projects to pull in other projects, including support for higher level projects 
being able to override dependency details of projects lower in the dependency 
tree if they want to. I've been incubating this privately in  a real world 
company environment to iron out the kinks and ensure the interface that 
projects interact with supports the right set of features and that less 
knowledgable users can understand it, etc. I'm planning on putting both 
DownloadProject and the dependency  framework up for review to merge into CMake 
within the next few months, all going well. Whereas your work seems to focus on 
building and/or re-using an installed/binary result, my work focuses more on 
making external projects part of your main build. This  has advantages like 
using the same compiler settings, toolchain details, etc. and all of the 
dependency targets are automatically available to the rest of the project if 
the dependency uses CMake as its build system. I'll postpone further details on 
it until  it is ready for review, but that should be enough for context for my 
responses to your comments further below.


There's also hunter, which you've already been made aware of and which is 
closer to your proposal in terms of the set of problems it tries to solve. 
Hunter has been around for a while, is reasonably well known  and on the face 
of it, already seems to do all the things cmake-get is trying to do. It is also 
based on ExternalProject. It may be helpful if you could show how your work 
differs from what hunter already provides so that its value to the CMake 
community is  clear.


Hopefully that's not too overwhelming for background/context. See my responses 
in the remainder below.



 


On Sat, Aug 5, 2017 at 5:57 AM, paul via CMake   wrote:
 Hi,

I have a written a cmake module to get dependencies using the cget protocol
here:

https://github.com/pfultz2/cmake-get

This is different than `ExternelProject`:

* `ExternelProject` happens only during build, which allows this module to
work in both in config and script mode. 



As detailed above, you can use ExternalProject at configure time too and 
there's already a relatively simple, generic implementation available showing 
how to do it for the download case.


   * In config mode, the user can just do:

cmake_get( PREFIX ${CMAKE_CURRENT_BINARY_DIR}/deps)
find_package( HINTS ${CMAKE_CURRENT_BINARY_DIR}/deps)

Since the dependency is available at config time.



This is a little naive. What if there is a complex hierarc

Re: [CMake] Interest in adding to CMake the CMakeGet module to get dependencies

[Craig Scott]

TLDR: There may be other work already covering similar capabilities (e.g.
hunter and WIP that I'm uncloaking here). This response is mostly about
providing broader discussion.


Paul it seems you've been busy since you first asked about cget
 on this list
last year. ;)

In the interests of disclosure, this latest email of yours has prompted me
to uncloak some of my own work which may be relevant. This work has been in
development and testing on real world projects for the past 2 years or so.
Without wanting to hijack your thread, let me give a little info about it
and then see my responses to your comments at the end for how/why I think
this may be relevant to your request for comments.

ExternalProject can be used in a different way to allow it to perform
downloads at configure time rather than build time. You can see a
relatively straightforward implementation of this here:

https://github.com/Crascit/DownloadProject

This has the advantage of leveraging all of the various download methods
ExternalProject already provides, meaning they are already fairly widely
used and tested. Any improvements to ExternalProject also come for free and
documentation for the various methods is also provided (I recently
overhauled the ExternalProject docs and these will appear in the 3.10.0
release, but they can be viewed in the master docs
 in
the meantime). ExternalProject also has the advantage that it supports
updating after the initial download in the case of repository sources like
git, svn, etc.

The uncloaking part of my response is that I have been working on a full
dependency download framework over the past year and half which is built on
top of the above DownloadProject implementation. It supports hierarchical
dependencies across complex project structures and makes it very easy for
projects to pull in other projects, including support for higher level
projects being able to override dependency details of projects lower in the
dependency tree if they want to. I've been incubating this privately in a
real world company environment to iron out the kinks and ensure the
interface that projects interact with supports the right set of features
and that less knowledgable users can understand it, etc. I'm planning on
putting both DownloadProject and the dependency framework up for review to
merge into CMake within the next few months, all going well. Whereas your
work seems to focus on building and/or re-using an installed/binary result,
my work focuses more on making external projects part of your main build.
This has advantages like using the same compiler settings, toolchain
details, etc. and all of the dependency targets are automatically available
to the rest of the project if the dependency uses CMake as its build
system. I'll postpone further details on it until it is ready for review,
but that should be enough for context for my responses to your comments
further below.

There's also hunter , which you've already
been made aware of and which is closer to your proposal in terms of the set
of problems it tries to solve. Hunter has been around for a while, is
reasonably well known and on the face of it, already seems to do all the
things cmake-get is trying to do. It is also based on ExternalProject. It
may be helpful if you could show how your work differs from what hunter
already provides so that its value to the CMake community is clear.

Hopefully that's not too overwhelming for background/context. See my
responses in the remainder below.



On Sat, Aug 5, 2017 at 5:57 AM, paul via CMake  wrote:

> Hi,
>
> I have a written a cmake module to get dependencies using the cget protocol
> here:
>
> https://github.com/pfultz2/cmake-get
>
> This is different than `ExternelProject`:
>
> * `ExternelProject` happens only during build, which allows this module to
> work in both in config and script mode.
>

As detailed above, you can use ExternalProject at configure time too and
there's already a relatively simple, generic implementation available
showing how to do it for the download case.



> * In config mode, the user can just do:
>
> cmake_get( PREFIX ${CMAKE_CURRENT_BINARY_DIR}/deps)
> find_package( HINTS ${CMAKE_CURRENT_BINARY_DIR}/deps)
>
> Since the dependency is available at config time.
>

This is a little naive. What if there is a complex hierarchy of projects
and more than one wants to use a particular dependency, but they both want
to specify different versions or repository details? How would that be
resolved? The "get" functionality looks quite limited, both in terms of the
supported methods and the ability to specify method-specific details.
ExternalProject already has pretty good coverage for this. In the case of
multiple projects wanting different dependency details, you need to decide
how to choose one project's details over another, or if y