Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Charles Huet]

Hi,

I was looking at tools that can do this kind of things myself (however I
was more looking at pre-built binaries redistribution than at a
super-build, since our build time is already quite long).

Does Conan (https://conan.io/) not fit your bill as well ?

Best

Le dim. 14 août 2016 à 02:33, Ian Henriksen <
insertinterestingnameh...@gmail.com> a écrit :

> On Sat, Aug 13, 2016 at 6:43 PM Elizabeth A. Fischer <
> elizabeth.fisc...@columbia.edu> wrote:
>
>> I would look into Anaconda, which does work for Windows.  Its version
>> management is not as well developed as Spack, but it's more cross-platform.
>>
>> Auto-builders are just coming into their own, it's a brave new world.  I
>> expect things to be more complete in a few years.
>>
>> -- Elizabeth
>>
>>
> Yes, Anaconda comes with a package manager (conda) that is designed to
> manage binary dependencies. You can get a minimal Python install with just
> the
> package manager by installing miniconda. The package management system is
> built in Python, but it can manage C and C++ libraries as well. The
> support for
> ARM isn't thoroughly built out, so you'll be building all the packages
> there for
> yourself. The support for Windwos, OS X and Linux is pretty well
> developed. You
> can look at https://repo.continuum.io/pkgs/ to see the packages available
> by
> default. There is also support for people uploading their own "channels"
> for
> packages on anaconda.org. The conda-forge channel in particular has a lot
> of
> additional packages.
> Hopefully that's more like what you're looking for.
> Best,
> Ian Henriksen
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Ian Henriksen]

On Sat, Aug 13, 2016 at 6:43 PM Elizabeth A. Fischer <
elizabeth.fisc...@columbia.edu> wrote:

> I would look into Anaconda, which does work for Windows.  Its version
> management is not as well developed as Spack, but it's more cross-platform.
>
> Auto-builders are just coming into their own, it's a brave new world.  I
> expect things to be more complete in a few years.
>
> -- Elizabeth
>
>
Yes, Anaconda comes with a package manager (conda) that is designed to
manage binary dependencies. You can get a minimal Python install with just
the
package manager by installing miniconda. The package management system is
built in Python, but it can manage C and C++ libraries as well. The support
for
ARM isn't thoroughly built out, so you'll be building all the packages
there for
yourself. The support for Windwos, OS X and Linux is pretty well developed.
You
can look at https://repo.continuum.io/pkgs/ to see the packages available by
default. There is also support for people uploading their own "channels" for
packages on anaconda.org. The conda-forge channel in particular has a lot of
additional packages.
Hopefully that's more like what you're looking for.
Best,
Ian Henriksen
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Elizabeth A. Fischer]

I would look into Anaconda, which does work for Windows.  Its version
management is not as well developed as Spack, but it's more cross-platform.

Auto-builders are just coming into their own, it's a brave new world.  I
expect things to be more complete in a few years.

-- Elizabeth


On Sat, Aug 13, 2016 at 7:00 PM, Robert Dailey 
wrote:

> I did some brief digging into spack, and it doesn't look like it
> supports Windows. All I see are shell scripts and the documentation
> uses POSIX.
>
> If I'm going to use a package manager, it needs to be able to support
> Android (ARM), Windows, and Linux. I have specific toolchains that
> I'll need the package manager to use for each of these platforms,
> assuming it compiles these libraries. It also needs to be capable of
> cross-compiling (in the case of ARM toolchain in the Android NDK).
>
> I mean, we can't really call it "reinventing the wheel" if my
> requirements are so specific that no current tooling can support it.
> If you have any info on spack related to my requirements please let me
> know. It looks promising, but so far doesn't seem like it will work
> out.
>
> On Fri, Aug 12, 2016 at 3:41 PM, Elizabeth A. Fischer
>  wrote:
> > This is what Spack and other meta builders do.  See http://github.
> > com/llnl/spack
> >
> > On Aug 12, 2016 3:59 PM, "Robert Dailey" 
> wrote:
> >>
> >> Hello,
> >>
> >> There is an internal C++ product at the company I work for which I
> >> have written a series of CMake scripts for. This project actually has
> >> dependencies on several open source libraries, such as boost,
> >> freetype, openssl, etc.
> >>
> >> Right now what we do is build each of these third party libraries *by
> >> hand*, once for every platform we support (Windows, Linux x86, Android
> >> NDK). Then we stuff the includes (headers) and libraries
> >> (static/shared) in a submodule and the primary code base's CMake
> >> scripts pull them in as interface targets.
> >>
> >> This works well and is light-weight but is a pain when upgrading or
> >> changing libraries. It's a pain because if I want to upgrade boost, I
> >> have to build it up to 6 times (once for each platform and once for
> >> each configuration).
> >>
> >> I've been thinking of a different approach for a while. I've done some
> >> toying around with the "Super Build" concept, where I have a separate
> >> CMake project that does nothing but use the ExternalProject module to
> >> build libraries in real time along with our project. So the order of
> >> operations would be as follows (for our automated build server):
> >>
> >> 1. Clone our "Third Party" repository
> >> 2. Use CMake to generate & build the "Super Build" project (this
> >> builds boost, openssl, freetype, etc for the current platform).
> >> 3. Clone the main code base's repository
> >> 4. Use CMake to generate & build, using find_package() to refer to
> >> interface targets exported by those third party libraries built in
> >> step 2
> >>
> >> Obviously this will make builds take significantly longer, because
> >> we're constantly rebuilding the same third party libraries over and
> >> over again. However, it virtually eliminates the maintenance burden
> >> for third party libraries because they are built inherently with
> >> everything else.
> >>
> >> Note that I can't refer to pre-built libraries in our build
> >> environments because we need very specific control over the versions
> >> of our libraries as well as the toolchains that were used to build
> >> them. Also we may specifically build our libraries a certain way (such
> >> as boost). For this reason we do not rely on our external environment
> >> or external package managers to fulfill third party dependencies, like
> >> most open source projects do on Linux for example.
> >>
> >> Does this "Super Build" approach sound like a better idea? What other
> >> options are available? The downside with the "Super Build" solution is
> >> that it will become very difficult to make the transition between
> >> building third party and building our code base seamless. I can't do
> >> both in the same generate step because find_package() can't be called
> >> until the libraries are built & installed.
> >> --
> >>
> >> Powered by www.kitware.com
> >>
> >> Please keep messages on-topic and check the CMake FAQ at:
> >> http://www.cmake.org/Wiki/CMake_FAQ
> >>
> >> Kitware offers various services to support the CMake community. For more
> >> information on each offering, please visit:
> >>
> >> CMake Support: http://cmake.org/cmake/help/support.html
> >> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> >> CMake Training Courses: http://cmake.org/cmake/help/training.html
> >>
> >> Visit other Kitware open-source projects at
> >> http://www.kitware.com/opensource/opensource.html
> >>
> >> Follow this link to subscribe/unsubscribe:
> >> http://public.kitware.com/mailman/listinfo/cmake
>
-- 

Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Robert Dailey]

I did some brief digging into spack, and it doesn't look like it
supports Windows. All I see are shell scripts and the documentation
uses POSIX.

If I'm going to use a package manager, it needs to be able to support
Android (ARM), Windows, and Linux. I have specific toolchains that
I'll need the package manager to use for each of these platforms,
assuming it compiles these libraries. It also needs to be capable of
cross-compiling (in the case of ARM toolchain in the Android NDK).

I mean, we can't really call it "reinventing the wheel" if my
requirements are so specific that no current tooling can support it.
If you have any info on spack related to my requirements please let me
know. It looks promising, but so far doesn't seem like it will work
out.

On Fri, Aug 12, 2016 at 3:41 PM, Elizabeth A. Fischer
 wrote:
> This is what Spack and other meta builders do.  See http://github.
> com/llnl/spack
>
> On Aug 12, 2016 3:59 PM, "Robert Dailey"  wrote:
>>
>> Hello,
>>
>> There is an internal C++ product at the company I work for which I
>> have written a series of CMake scripts for. This project actually has
>> dependencies on several open source libraries, such as boost,
>> freetype, openssl, etc.
>>
>> Right now what we do is build each of these third party libraries *by
>> hand*, once for every platform we support (Windows, Linux x86, Android
>> NDK). Then we stuff the includes (headers) and libraries
>> (static/shared) in a submodule and the primary code base's CMake
>> scripts pull them in as interface targets.
>>
>> This works well and is light-weight but is a pain when upgrading or
>> changing libraries. It's a pain because if I want to upgrade boost, I
>> have to build it up to 6 times (once for each platform and once for
>> each configuration).
>>
>> I've been thinking of a different approach for a while. I've done some
>> toying around with the "Super Build" concept, where I have a separate
>> CMake project that does nothing but use the ExternalProject module to
>> build libraries in real time along with our project. So the order of
>> operations would be as follows (for our automated build server):
>>
>> 1. Clone our "Third Party" repository
>> 2. Use CMake to generate & build the "Super Build" project (this
>> builds boost, openssl, freetype, etc for the current platform).
>> 3. Clone the main code base's repository
>> 4. Use CMake to generate & build, using find_package() to refer to
>> interface targets exported by those third party libraries built in
>> step 2
>>
>> Obviously this will make builds take significantly longer, because
>> we're constantly rebuilding the same third party libraries over and
>> over again. However, it virtually eliminates the maintenance burden
>> for third party libraries because they are built inherently with
>> everything else.
>>
>> Note that I can't refer to pre-built libraries in our build
>> environments because we need very specific control over the versions
>> of our libraries as well as the toolchains that were used to build
>> them. Also we may specifically build our libraries a certain way (such
>> as boost). For this reason we do not rely on our external environment
>> or external package managers to fulfill third party dependencies, like
>> most open source projects do on Linux for example.
>>
>> Does this "Super Build" approach sound like a better idea? What other
>> options are available? The downside with the "Super Build" solution is
>> that it will become very difficult to make the transition between
>> building third party and building our code base seamless. I can't do
>> both in the same generate step because find_package() can't be called
>> until the libraries are built & installed.
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to 

Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Elizabeth A. Fischer]

>
> This is what Spack and other meta builders do.  I don't think CMake is the
> best place to do it, for a number of reasons.  I would not try to re-invent
> the wheel here.
>
See http://github. com/llnl/spack
>

-- Elizabeth
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers