[cmake-developers] Appx generator for CPack?

[Ryan Saunders via cmake-developers]

Hi CMake people!
I notice that there is no CPack generator for Microsoft's .appx package format. 
I would like there to be. It seems pretty straightforward to do...basically 
just get all the files into the right directory structure (similarly to the 
archive generators, I think), and then call out to Visual Studio's MakeAppx.exe 
to create the package.
Two questions:1. Is anyone already working on this? If not, then I'll attempt 
it.2. Am I missing anything? Any "gotchas" I should worry about?
Ryan-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] cmake -E capabilities [v3]

[Brad King]

On 08/16/2016 02:39 PM, Tobias Hunger wrote:
> Did you make any progress with libuv? I would love to start pushing
> the server-mode patches. All the other pieces are in place already!

Great.  I haven't gotten a chance to import libuv yet.  Now that
you're ready for it I'll try to raise priority on that a bit.

-Brad

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [PATCH v7] For Windows encode process output to internally used encoding

[Brad King]

On 08/15/2016 04:34 PM, Dāvis Mosāns wrote:
> Typically Windows applications (eg. MSVC compiler) use current console's
> codepage for output to pipes so we need to encode that to internally used
> encoding (KWSYS_ENCODING_DEFAULT_CODEPAGE).

Thanks.  Applied with minor tweaks and merged to `next` for testing:

 Windows: Encode child process output to internally-used encoding
 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=0a818239

-Brad
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] llvm coverage format .profdata

[Brad King]

On 08/15/2016 02:24 PM, Alex Ciobanu wrote:
> Are there any plans in this direction?

Not currently, but one can look at the implementations of the currently
supported formats to add it.

Thanks,
-Brad

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] cmake -E capabilities [v3]

[Brad King]

On 08/15/2016 09:56 AM, Tobias Hunger wrote:
> here is a new update of the cmake -E capabilities patch.

Thanks.  Applied with minor revisions and an additional test:

 cmake: Add `cmake -E capabilities` mode
 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=49ad7f9a

-Brad

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [PATCH] cmFileCommand: Download continuation support

[Brad King]

On 08/16/2016 07:11 AM, Titov Denis wrote:
> This patch introduces several options for the file(DOWNLOAD ...) command, 
> which
> would be useful in case of dealing with big files or unstable network 
> connections.

Nice, thanks.  Good start.  Here are a few minor comments.

> The added options are:

Please also update `Help/command/file.rst` to document these.

> +void crossplatform_sleep(int delay_s)

Please use cmSystemTools::Delay instead.

Thanks,
-Brad
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Need ideas/opinions on third party library management

[Florent Castelli]

We have many smaller libraries and a few big ones like Boost.
It certainly takes a bit of time to setup, but it will save you a lot of time 
later.
It will keep your build scripts clean too as the user won’t have to know about
its dependencies setup. Just use “target_link_libraries” on the modern target
that was created and it will work.

Most of the 3rd party dependencies don’t work that way and they're quite painful
to use correctly. As in, if you miss to add the “add_definitions” they told you 
to use,
it will build, but source files might be used with a different preprocessor 
define and
you end up with two different implementations of the same class in the same 
binary.
It will crash in very mysterious ways. 

The most important point is *correctness*. If you experiment with tooling like
the Clang sanitizers, you’ll know that you can’t mix libraries that haven’t been
instrumented with instrumented code.
Best case the libraries are self-contained and will work all of the time.
Worst case, the libraries are mostly self-contained and will *not* work 
sometimes!

As an example, the Boost Unit Test Framework usually worked fine for us.
But sometimes, we had ASan triggering an issue in the constructor of the string
object for the test name. Our binary was built with ASan and Boost without.
When we built it from source, all the false positives went away.

I know that not a lot of people use them or have heard of them, but they are
a super important tool that will find TONS of real issues in your code the first
time you use them and we cannot do without.
Some people might rely on Valgrind Memcheck instead, but hey, this is 2016
and we like our tests to run fast(er)!


/Florent

> On 16 Aug 2016, at 14:41, Benjamin Ballet  wrote:
> 
> Very interesting discussion, we have the same issues here.
> 
> Florent Castelli, how many third parties libraries do you use ? I think a 
> super build can be a very good solution but I'm wondering how much third 
> party code you have to build. Here we use OpenCV, with, boost, and poco, and 
> other things... So it may be too long.
> 
> I was personnaly considering having an hybrid solution : include small 
> libraries (like jsoncpp) and pre-build the other for each platforms.
> 
> 
> 2016-08-16 12:52 GMT+02:00 Florent Castelli  >:
> At Spotify, we use CMake a lot for our large C++ library shared by all the 
> clients.
> After trying to build libraries for each platform and variant, we basically 
> gave up and we now
> use a super-build approach.
> 
> For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android and 
> iOS.
> Each platform has a different CPU target (or many 32/64bit, x86/ARM).
> Each platform has many compilers.
> Some platforms have instrumentation options (Debug / Release, ASan, MSan…) 
> and really need
> to be compiled properly, otherwise you’ll end up with false positives.
> The matrix of builds is REALLY hard to track. Each time we update Boost, we 
> had to update
> a lot of things.
> I tried using ExternalProject and use b2 (build tool from Boost) to build it 
> and instead of having
> lots of build jobs with a mirror of the flags, you end up mirroring the flags 
> in your CMake files
> instead, which is still not good enough.
> 
> In the end, I looked at how Boost is actually built. And for most libraries, 
> it’s plain simple.
> A static library with a few files, some define, sometimes a platform specific 
> source file.
> What if instead of having an external build tool, I built it from CMake 
> instead?
> It would propagate all the build flags, target, instrumentation and compiler 
> information from the main
> build to it and just work.
> I tried it and it worked in no time! We replaced our Boost 1.59 binary 
> distribution with the source
> distribution and it’s much easier. When people build our library for a 
> different target, they don’t have
> to download new binaries, they just reuse the same sources.
> Later on, we found a bug in Boost 1.59 (fixed in later versions) and patched 
> it. We updated our source
> bundle and everything was smooth.
> Much later on, we wanted to use 1.61. We just updated the source bundle 
> again, the list of source
> files or compilation flags for the libraries we use didn’t change. It was 
> again effortless.
> 
> Overall, building boost takes 10s on our developers’ machines. The sources 
> aren’t changed often,
> so the cost is pretty low. It needs attention when we upgrade it, but that’s 
> quite rare.
> 
> We try now to use the same approach for other libraries when we add them. 
> Some of them are
> already using CMake and it’s somewhat easier, but since most people still 
> target version 2.8 (or 2.6...),
> we find it better to rewrite the build scripts ourselves and use modern 
> features (as in, everything is
> a target that propagates requirements, we don’t propagate variables).
> It makes it also much easier 

Re: [cmake-developers] Need ideas/opinions on third party library management

[Elizabeth A. Fischer]

CMake builds for existing libraries are certainly an interesting and useful
thing, and deserve to be posted in a GitHub repo somewhere.  They should
also serve as the basis of a campaign to get the library authors to
incorporate the CMake build directly in their repos.

But any approach that requires every build to be ported to CMake will be
difficult and labor-prone to scale.  Writing a meta-build recipe is usually
much easier.

Spack handles the combinatorial dependencies you mention in a
sophisticated, graceful way that most meta-builders do not.  Its only
problem is it does not (yet) run on Windows.  There's no fundamental reason
why not; we just need someone to get involved and start trying it on
Windows.

-- Elizabeth


On Tue, Aug 16, 2016 at 6:52 AM, Florent Castelli <
florent.caste...@gmail.com> wrote:

> At Spotify, we use CMake a lot for our large C++ library shared by all the
> clients.
> After trying to build libraries for each platform and variant, we
> basically gave up and we now
> use a super-build approach.
>
> For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android
> and iOS.
> Each platform has a different CPU target (or many 32/64bit, x86/ARM).
> Each platform has many compilers.
> Some platforms have instrumentation options (Debug / Release, ASan, MSan…)
> and really need
> to be compiled properly, otherwise you’ll end up with false positives.
> The matrix of builds is REALLY hard to track. Each time we update Boost,
> we had to update
> a lot of things.
> I tried using ExternalProject and use b2 (build tool from Boost) to build
> it and instead of having
> lots of build jobs with a mirror of the flags, you end up mirroring the
> flags in your CMake files
> instead, which is still not good enough.
>
> In the end, I looked at how Boost is actually built. And for most
> libraries, it’s plain simple.
> A static library with a few files, some define, sometimes a platform
> specific source file.
> What if instead of having an external build tool, I built it from CMake
> instead?
> It would propagate all the build flags, target, instrumentation and
> compiler information from the main
> build to it and just work.
> I tried it and it worked in no time! We replaced our Boost 1.59 binary
> distribution with the source
> distribution and it’s much easier. When people build our library for a
> different target, they don’t have
> to download new binaries, they just reuse the same sources.
> Later on, we found a bug in Boost 1.59 (fixed in later versions) and
> patched it. We updated our source
> bundle and everything was smooth.
> Much later on, we wanted to use 1.61. We just updated the source bundle
> again, the list of source
> files or compilation flags for the libraries we use didn’t change. It was
> again effortless.
>
> Overall, building boost takes 10s on our developers’ machines. The sources
> aren’t changed often,
> so the cost is pretty low. It needs attention when we upgrade it, but
> that’s quite rare.
>
> We try now to use the same approach for other libraries when we add them.
> Some of them are
> already using CMake and it’s somewhat easier, but since most people still
> target version 2.8 (or 2.6...),
> we find it better to rewrite the build scripts ourselves and use modern
> features (as in, everything is
> a target that propagates requirements, we don’t propagate variables).
> It makes it also much easier to build a library for another platform that
> wasn’t targeted by the original
> project.
>
> If people are interested, I could share the CMakeLists.txt file we use for
> Boost. It doesn’t build all
> the libraries (some are hard like Context) and uses some internal macros,
> but it should be plain
> simple to tweak for your use.
>
> /Florent
>
> > On 12 Aug 2016, at 21:59, Robert Dailey 
> wrote:
> >
> > Hello,
> >
> > There is an internal C++ product at the company I work for which I
> > have written a series of CMake scripts for. This project actually has
> > dependencies on several open source libraries, such as boost,
> > freetype, openssl, etc.
> >
> > Right now what we do is build each of these third party libraries *by
> > hand*, once for every platform we support (Windows, Linux x86, Android
> > NDK). Then we stuff the includes (headers) and libraries
> > (static/shared) in a submodule and the primary code base's CMake
> > scripts pull them in as interface targets.
> >
> > This works well and is light-weight but is a pain when upgrading or
> > changing libraries. It's a pain because if I want to upgrade boost, I
> > have to build it up to 6 times (once for each platform and once for
> > each configuration).
> >
> > I've been thinking of a different approach for a while. I've done some
> > toying around with the "Super Build" concept, where I have a separate
> > CMake project that does nothing but use the ExternalProject module to
> > build libraries in real time along with our project. So the order of
> > operations would be 

Re: [cmake-developers] Need ideas/opinions on third party library management

[Benjamin Ballet via cmake-developers]

Very interesting discussion, we have the same issues here.

Florent Castelli, how many third parties libraries do you use ? I think a
super build can be a very good solution but I'm wondering how much third
party code you have to build. Here we use OpenCV, with, boost, and poco,
and other things... So it may be too long.

I was personnaly considering having an hybrid solution : include small
libraries (like jsoncpp) and pre-build the other for each platforms.


2016-08-16 12:52 GMT+02:00 Florent Castelli :

> At Spotify, we use CMake a lot for our large C++ library shared by all the
> clients.
> After trying to build libraries for each platform and variant, we
> basically gave up and we now
> use a super-build approach.
>
> For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android
> and iOS.
> Each platform has a different CPU target (or many 32/64bit, x86/ARM).
> Each platform has many compilers.
> Some platforms have instrumentation options (Debug / Release, ASan, MSan…)
> and really need
> to be compiled properly, otherwise you’ll end up with false positives.
> The matrix of builds is REALLY hard to track. Each time we update Boost,
> we had to update
> a lot of things.
> I tried using ExternalProject and use b2 (build tool from Boost) to build
> it and instead of having
> lots of build jobs with a mirror of the flags, you end up mirroring the
> flags in your CMake files
> instead, which is still not good enough.
>
> In the end, I looked at how Boost is actually built. And for most
> libraries, it’s plain simple.
> A static library with a few files, some define, sometimes a platform
> specific source file.
> What if instead of having an external build tool, I built it from CMake
> instead?
> It would propagate all the build flags, target, instrumentation and
> compiler information from the main
> build to it and just work.
> I tried it and it worked in no time! We replaced our Boost 1.59 binary
> distribution with the source
> distribution and it’s much easier. When people build our library for a
> different target, they don’t have
> to download new binaries, they just reuse the same sources.
> Later on, we found a bug in Boost 1.59 (fixed in later versions) and
> patched it. We updated our source
> bundle and everything was smooth.
> Much later on, we wanted to use 1.61. We just updated the source bundle
> again, the list of source
> files or compilation flags for the libraries we use didn’t change. It was
> again effortless.
>
> Overall, building boost takes 10s on our developers’ machines. The sources
> aren’t changed often,
> so the cost is pretty low. It needs attention when we upgrade it, but
> that’s quite rare.
>
> We try now to use the same approach for other libraries when we add them.
> Some of them are
> already using CMake and it’s somewhat easier, but since most people still
> target version 2.8 (or 2.6...),
> we find it better to rewrite the build scripts ourselves and use modern
> features (as in, everything is
> a target that propagates requirements, we don’t propagate variables).
> It makes it also much easier to build a library for another platform that
> wasn’t targeted by the original
> project.
>
> If people are interested, I could share the CMakeLists.txt file we use for
> Boost. It doesn’t build all
> the libraries (some are hard like Context) and uses some internal macros,
> but it should be plain
> simple to tweak for your use.
>
> /Florent
>
> > On 12 Aug 2016, at 21:59, Robert Dailey 
> wrote:
> >
> > Hello,
> >
> > There is an internal C++ product at the company I work for which I
> > have written a series of CMake scripts for. This project actually has
> > dependencies on several open source libraries, such as boost,
> > freetype, openssl, etc.
> >
> > Right now what we do is build each of these third party libraries *by
> > hand*, once for every platform we support (Windows, Linux x86, Android
> > NDK). Then we stuff the includes (headers) and libraries
> > (static/shared) in a submodule and the primary code base's CMake
> > scripts pull them in as interface targets.
> >
> > This works well and is light-weight but is a pain when upgrading or
> > changing libraries. It's a pain because if I want to upgrade boost, I
> > have to build it up to 6 times (once for each platform and once for
> > each configuration).
> >
> > I've been thinking of a different approach for a while. I've done some
> > toying around with the "Super Build" concept, where I have a separate
> > CMake project that does nothing but use the ExternalProject module to
> > build libraries in real time along with our project. So the order of
> > operations would be as follows (for our automated build server):
> >
> > 1. Clone our "Third Party" repository
> > 2. Use CMake to generate & build the "Super Build" project (this
> > builds boost, openssl, freetype, etc for the current platform).
> > 3. Clone the main code base's repository
> > 4. Use 

[cmake-developers] [PATCH] cmFileCommand: Download continuation support

[Titov Denis]

This patch introduces several options for the file(DOWNLOAD ...) command, which
would be useful in case of dealing with big files or unstable network
connections.

The added options are:

RETRY_COUNT  -- sets maximal amount of download restarts, default value: 1

RETRY_DELAY  -- sets delay before restarting download (in seconds),
default value: 0.0

RETRY_MAX_TIME  -- sets maximal time spent in downloading file
(in seconds), default value: infinity

RETRY_CONTINUE -- if set, makes cmake try to continue downloading of the
existing chunk, instead of discarding it and starting all over. This option is
not set by default

Notes:

The RETRY_CONTINUE option requires server-side support of http partial get
(content-range header).

Unfortunately, I haven't been able to properly test the RETRY_CONTINUE option,
as I didn't have access to the appropriate server. Any help in this area is
encouraged.
---
 Source/cmFileCommand.cxx | 287 +--
 1 file changed, 204 insertions(+), 83 deletions(-)

diff --git a/Source/cmFileCommand.cxx b/Source/cmFileCommand.cxx
index 835b118..aa7782e 100644
--- a/Source/cmFileCommand.cxx
+++ b/Source/cmFileCommand.cxx
@@ -34,6 +34,16 @@
 // include sys/stat.h after sys/types.h
 #include 
 
+#include 
+#include 
+
+// For crossplatform_sleep().
+#if defined(_WIN32) && !defined(__CYGWIN__)
+#include 
+#else
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -68,6 +78,15 @@ static mode_t mode_setuid = S_ISUID;
 static mode_t mode_setgid = S_ISGID;
 #endif
 
+void crossplatform_sleep(int delay_s)
+{
+#if defined(_WIN32) && !defined(__CYGWIN__)
+  Sleep(delay_s);
+#else
+  sleep(delay_s * 1000);
+#endif
+}
+
 #if defined(_WIN32) && defined(CMAKE_ENCODING_UTF8)
 // libcurl doesn't support file:// urls for unicode filenames on Windows.
 // Convert string from UTF-8 to ACP if this is a file:// URL.
@@ -2481,6 +2500,11 @@ bool 
cmFileCommand::HandleDownloadCommand(std::vector const& args)
   std::string hashMatchMSG;
   CM_AUTO_PTR hash;
   bool showProgress = false;
+  int retryMaxCount = 1;
+  double retryDelayS = 0.0;
+  double retryMaxTimeS = DBL_MAX;
+  bool retryContinue = false;
+  cmsys::ofstream fout;
 
   while (i != args.end()) {
 if (*i == "TIMEOUT") {
@@ -2564,7 +2588,34 @@ bool 
cmFileCommand::HandleDownloadCommand(std::vector const& args)
 return false;
   }
   hashMatchMSG = algo + " hash";
+} else if (*i == "RETRY_COUNT") {
+  ++i;
+  if (i != args.end()) {
+retryMaxCount = atoi(i->c_str());
+  } else {
+this->SetError("DOWNLOAD missing count for RETRY_COUNT");
+return false;
+  }
+} else if (*i == "RETRY_DELAY") {
+  ++i;
+  if (i != args.end()) {
+retryDelayS = atof(i->c_str());
+  } else {
+this->SetError("DOWNLOAD missing time for RETRY_DELAY");
+return false;
+  }
+} else if (*i == "RETRY_MAX_TIME") {
+  ++i;
+  if (i != args.end()) {
+retryMaxTimeS = atof(i->c_str());
+  } else {
+this->SetError("DOWNLOAD missing time for RETRY_MAX_TIME");
+return false;
+  }
+} else if (*i == "RETRY_CONTINUE") {
+  retryContinue = true;
 }
+
 ++i;
   }
   // If file exists already, and caller specified an expected md5 or sha,
@@ -2599,110 +2650,171 @@ bool 
cmFileCommand::HandleDownloadCommand(std::vector const& args)
 return false;
   }
 
-  cmsys::ofstream fout(file.c_str(), std::ios::binary);
-  if (!fout) {
-this->SetError("DOWNLOAD cannot open file for write.");
-return false;
-  }
-
 #if defined(_WIN32) && defined(CMAKE_ENCODING_UTF8)
   url = fix_file_url_windows(url);
 #endif
 
+  cmFileCommandVectorOfChar chunkDebug;
+
   ::CURL* curl;
   ::curl_global_init(CURL_GLOBAL_DEFAULT);
-  curl = ::curl_easy_init();
-  if (!curl) {
-this->SetError("DOWNLOAD error initializing curl.");
-return false;
-  }
 
-  cURLEasyGuard g_curl(curl);
-  ::CURLcode res = ::curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
-  check_curl_result(res, "DOWNLOAD cannot set url: ");
+  ::CURLcode res;
+  int tries = 0;
+  double elapsed = 0.0;
+  time_t start, end;
+  while (tries < retryMaxCount && elapsed <= retryMaxTimeS) {
+++tries;
+time();
+
+curl = ::curl_easy_init();
+if (!curl) {
+  this->SetError("DOWNLOAD error initializing curl.");
+  ::curl_global_cleanup();
+  return false;
+}
 
-  // enable HTTP ERROR parsing
-  res = ::curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1);
-  check_curl_result(res, "DOWNLOAD cannot set http failure option: ");
+if (cmSystemTools::FileExists(file.c_str())) { // Something was downloaded.
+  // Check hash.
+  if (hash.get()) {
+std::string actualHash = hash->HashFile(file);
+if (actualHash == expectedHash) { // File is complete, exit.
+  ::curl_easy_cleanup(curl);
+  ::curl_global_cleanup();
+  return true;
+}
+  }
 
-  res = 

Re: [cmake-developers] Need ideas/opinions on third party library management

[Florent Castelli]

At Spotify, we use CMake a lot for our large C++ library shared by all the 
clients.
After trying to build libraries for each platform and variant, we basically 
gave up and we now
use a super-build approach.

For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android and iOS.
Each platform has a different CPU target (or many 32/64bit, x86/ARM).
Each platform has many compilers.
Some platforms have instrumentation options (Debug / Release, ASan, MSan…) and 
really need
to be compiled properly, otherwise you’ll end up with false positives.
The matrix of builds is REALLY hard to track. Each time we update Boost, we had 
to update
a lot of things.
I tried using ExternalProject and use b2 (build tool from Boost) to build it 
and instead of having
lots of build jobs with a mirror of the flags, you end up mirroring the flags 
in your CMake files
instead, which is still not good enough.

In the end, I looked at how Boost is actually built. And for most libraries, 
it’s plain simple.
A static library with a few files, some define, sometimes a platform specific 
source file.
What if instead of having an external build tool, I built it from CMake instead?
It would propagate all the build flags, target, instrumentation and compiler 
information from the main
build to it and just work.
I tried it and it worked in no time! We replaced our Boost 1.59 binary 
distribution with the source
distribution and it’s much easier. When people build our library for a 
different target, they don’t have
to download new binaries, they just reuse the same sources.
Later on, we found a bug in Boost 1.59 (fixed in later versions) and patched 
it. We updated our source
bundle and everything was smooth.
Much later on, we wanted to use 1.61. We just updated the source bundle again, 
the list of source
files or compilation flags for the libraries we use didn’t change. It was again 
effortless.

Overall, building boost takes 10s on our developers’ machines. The sources 
aren’t changed often,
so the cost is pretty low. It needs attention when we upgrade it, but that’s 
quite rare.

We try now to use the same approach for other libraries when we add them. Some 
of them are
already using CMake and it’s somewhat easier, but since most people still 
target version 2.8 (or 2.6...),
we find it better to rewrite the build scripts ourselves and use modern 
features (as in, everything is
a target that propagates requirements, we don’t propagate variables).
It makes it also much easier to build a library for another platform that 
wasn’t targeted by the original
project.

If people are interested, I could share the CMakeLists.txt file we use for 
Boost. It doesn’t build all
the libraries (some are hard like Context) and uses some internal macros, but 
it should be plain
simple to tweak for your use.

/Florent

> On 12 Aug 2016, at 21:59, Robert Dailey  wrote:
> 
> Hello,
> 
> There is an internal C++ product at the company I work for which I
> have written a series of CMake scripts for. This project actually has
> dependencies on several open source libraries, such as boost,
> freetype, openssl, etc.
> 
> Right now what we do is build each of these third party libraries *by
> hand*, once for every platform we support (Windows, Linux x86, Android
> NDK). Then we stuff the includes (headers) and libraries
> (static/shared) in a submodule and the primary code base's CMake
> scripts pull them in as interface targets.
> 
> This works well and is light-weight but is a pain when upgrading or
> changing libraries. It's a pain because if I want to upgrade boost, I
> have to build it up to 6 times (once for each platform and once for
> each configuration).
> 
> I've been thinking of a different approach for a while. I've done some
> toying around with the "Super Build" concept, where I have a separate
> CMake project that does nothing but use the ExternalProject module to
> build libraries in real time along with our project. So the order of
> operations would be as follows (for our automated build server):
> 
> 1. Clone our "Third Party" repository
> 2. Use CMake to generate & build the "Super Build" project (this
> builds boost, openssl, freetype, etc for the current platform).
> 3. Clone the main code base's repository
> 4. Use CMake to generate & build, using find_package() to refer to
> interface targets exported by those third party libraries built in
> step 2
> 
> Obviously this will make builds take significantly longer, because
> we're constantly rebuilding the same third party libraries over and
> over again. However, it virtually eliminates the maintenance burden
> for third party libraries because they are built inherently with
> everything else.
> 
> Note that I can't refer to pre-built libraries in our build
> environments because we need very specific control over the versions
> of our libraries as well as the toolchains that were used to build
> them. Also we may specifically build our libraries a certain 

Re: [cmake-developers] [Patch 5/5] Improved WIX support

[Nils Gladitz]

On 08/16/2016 11:15 AM, Stuermer, Michael SP/HZA-ZSEP wrote:


There is precedence in cmWIXFilesSourceWriter::EmitComponentFile() so I
think such an interface change would be fine.


Ok I'll do this. Should solve all issues and doubts hopefully.


Great. Thanks.



  Adding FeatureRef to #PRODUCT does not work. I get the following message:

features.wixobj : error LGHT0095 : Multiple primary references were found for 
Feature '@feature@' in Feature 'ProductFeature' and Product '{@guid@}'


Did you try with IgnoreParent="yes" on your FeatureRef element?
It makes sense that there can only be one root-Feature.

Nils
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [Patch 5/5] Improved WIX support

[Stuermer, Michael SP/HZA-ZSEP]

> -Original Message-
> From: Nils Gladitz [mailto:nilsglad...@gmail.com]
> Sent: Tuesday, August 16, 2016 10:54 AM
> To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers
> Subject: Re: [cmake-developers] [Patch 5/5] Improved WIX support
> 
> On 08/16/2016 10:15 AM, Stuermer, Michael SP/HZA-ZSEP wrote:
> 
> >
> > After having  look at the code for some minutes I remember why patching
> the ref instead oft he feature was my way to go:
> >
> > The feature is written to file in
> cmWIXFeaturesSourceWriter::EmitFeatureForComponent(), where I do not
> have any patch information available. This means I'd have to change the
> signature of both EmitFeatureForComponent and
> EmitFeatureForComponentGroup and pass a reference to the patch instance
> along. Multiple occurrence of IDs can happen, but the patch will only be
> applied once because applied fragments are erased immediately after
> writing them to the stream.
> >
> > So after all for me this was a consideration of a 1-line change vs. changing
> class interfaces an passing object instances to where it might not be
> desirable.
> 
> There is precedence in cmWIXFilesSourceWriter::EmitComponentFile() so I
> think such an interface change would be fine.
> 

Ok I'll do this. Should solve all issues and doubts hopefully.

> >
> > I agree the commit message of the patch is not accurate enough and if
> there is way to add custom WIX-components to features without changing
> the cpack source I'd be happy to do so. But so far I tried several approaches
> and neither worked (see below).
> >
> >> This would not be any more convenient but would certainly match
> >> expectations and be less ill defined.
> >> e.g. when I specify a patch for a Feature I expect that the Feature
> >> with the given ID gets patched.
> >>
> >> Arguments against patching a FeatureRef instead are:
> >> - There can be n FeatureRef elements for any Feature element in which
> >> case it would not be obvious if the patch should be applied to one
> >> (which?) or all of these
> > The way the patch was implemented only the featurerefs in the generated
> features.wxs file would be patched and there should not be any double
> occurences of a feature ref.
> >
> >> - While similar FeatureRef and Feature don't take the same Child
> >> elements
> > Right, and if both Feature and FeatureRef would be patchable we would be
> in trouble. For the lazy one: this is not the case at the moment so we would
> not need to worry about it (but it's very nice). For the correct one: We could
... meant NOT very nice, of course ...
> introduce another attribute to CPackWixFragment called "Type" where type
> of the XML node to be patched could be stored. But this would introduce
> additional complexity to the cmWIXPatch class...
> 
> There is no use case to be able to patch both FeatureRef and Feature
> elements when Feature elements can be patched directly.
> 
Right.
> >
> >> - You can already define your own FeatureRef elements (referencing
> >> any of the pre-existing Feature elements if wanted) without having to
> >> use the patch mechanism
> >>
> > I tried this like this (in a separate additional .wxs source file added with
> CPACK_WIX_EXTRA_SOURCES):
> >
> > http://schemas.microsoft.com/wix/2006/wi;>
> >
> >   Directory="INSTALL_ROOT">
> >
> >  
> >
> >  
> >   IgnoreParent="yes">
> >
> >  
> >
> > 
> >
> > Did not work, the registry value was not set. Using the proposed approach
> it worked. Do I have to reference it somehow different?
> 
> The linker only includes object files which provide a symbol that is required
> by an object already included.
> Your source file has a single symbol for the Component "SetRegistryValues"
> but that symbol (I assume) is not required by any of the other objects which
> the linker includes.
> 
> You could e.g. add the FeatureRef to a custom WIX.template.in (which has
> the main entry point and is therefor always included), or supply a patch for
> the Product element (#PRODUCT), or create any kind of valid reference to
> your custom source file (if any reference is resolved through your custom
> source the entire object gets included).
> 

 Adding FeatureRef to #PRODUCT does not work. I get the following message:

features.wixobj : error LGHT0095 : Multiple primary references were found for 
Feature '@feature@' in Feature 'ProductFeature' and Product '{@guid@}'

> Nils
> .

Michael

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to 

Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Charles Huet]

Hi,

I was looking at tools that can do this kind of things myself (however I
was more looking at pre-built binaries redistribution than at a
super-build, since our build time is already quite long).

Does Conan (https://conan.io/) not fit your bill as well ?

Best

Le dim. 14 août 2016 à 02:33, Ian Henriksen <
insertinterestingnameh...@gmail.com> a écrit :

> On Sat, Aug 13, 2016 at 6:43 PM Elizabeth A. Fischer <
> elizabeth.fisc...@columbia.edu> wrote:
>
>> I would look into Anaconda, which does work for Windows.  Its version
>> management is not as well developed as Spack, but it's more cross-platform.
>>
>> Auto-builders are just coming into their own, it's a brave new world.  I
>> expect things to be more complete in a few years.
>>
>> -- Elizabeth
>>
>>
> Yes, Anaconda comes with a package manager (conda) that is designed to
> manage binary dependencies. You can get a minimal Python install with just
> the
> package manager by installing miniconda. The package management system is
> built in Python, but it can manage C and C++ libraries as well. The
> support for
> ARM isn't thoroughly built out, so you'll be building all the packages
> there for
> yourself. The support for Windwos, OS X and Linux is pretty well
> developed. You
> can look at https://repo.continuum.io/pkgs/ to see the packages available
> by
> default. There is also support for people uploading their own "channels"
> for
> packages on anaconda.org. The conda-forge channel in particular has a lot
> of
> additional packages.
> Hopefully that's more like what you're looking for.
> Best,
> Ian Henriksen
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers