Re: [cmake-developers] [ANNOUNCE] CMake 3.5.0-rc1 now ready for testing!

[James Johnston]

> -Original Message-
> From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
> On Behalf Of Brad King
> Sent: Wednesday, February 03, 2016 21:17
> To: Eric Wing; Robert Maynard
> Cc: CMake Developers
> Subject: Re: [cmake-developers] [ANNOUNCE] CMake 3.5.0-rc1 now ready
> for testing!
> 
> On 02/03/2016 03:48 PM, Eric Wing wrote:
> > Hi, I'm curious about this. Would you explain the difference between
> > the .msi and the installer?
> 
> The .exe was a NSIS-generated installer tool.  There were several bugs in
it
> caused by problems with NSIS, but worst is that it could wipe out long
PATH
> environment variables.  The .msi is more like a .deb or .rpm but that is
> managed by the Windows Installer Service.

There's several other reasons as well.  Not that I had anything to do with
CMake moving to WiX, but I'm happy to see they did:

* Windows Installer is transactional.  If there's an error or other problem
during installation, MSI will roll back the installation - it "just works"
for the most part.  Whereas with NSIS, offering something as simple as a
"Cancel" button is apparently quite difficult:
http://nsis.sourceforge.net/InstFiles_Cancel_-_Allowing_a_user_to_cancel_ins
tallation_during_InstFiles

* Self-healing functionality: if the application installation is damaged,
MSI can automatically repair it when launching the application.  If you
like, you can have it go as far as verify checksums of each file.

* MSI is declarative: standardized database schema defines what to install
and where - not how to install it.  So installs, uninstalls, repairs,
recovery from a failed transaction, etc. "just work".  The standardized
behavior could also help system administrators to customize the installation
with existing tools.

* MSI packages can be deployed with Active Directory, and non-administrator
users can elect to install them while the installation still runs at an
elevated level (or the installation can be forced when the system starts or
when the user logs in).

Nice discussion here: http://serverfault.com/a/274609

Maybe now that CPack supports WiX, more open source projects will start to
use it instead of NSIS. :)

I only hope they set up the MSIs to support side-by-side installations of
multiple CMake versions.  I have yet to find time to see if this is the
case.

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CMake daemon for user tools

[James Johnston]

> -Original Message-
> From: Milian Wolff [mailto:m...@milianw.de]
> Sent: Saturday, January 23, 2016 15:41
> To: cmake-developers@cmake.org
> Cc: James Johnston
> Subject: Re: [cmake-developers] CMake daemon for user tools
> 
> You are aware that modern std::string is SSO'ed? I'm running on such a
> system.
> Another reason why you should not reinvent the wheel and keep relying on
> the STL wherever possible.
>

> Qt has such a class, it's called QVarLengthArray, and I've also been able
to
> apply it in multiple occasions to good effect. That said, when you look at

Yeah, but std::string is platform dependent, and the size of the buffer is
also platform dependent.  Maybe it tends to be optimal for CMake.  Then
again, maybe a larger buffer is needed.  I don't know.  The flexible option
would be something that does exactly like QVarLengthArray.  Different
variables might have different optimal sizes.

Some sample small strings for gcc/clang/VC++:
http://stackoverflow.com/a/28003328/562766
Note that none of them are large enough to store an absolute path, which are
maybe common (???) in CMake.  Also there's a fair bit of variation; if CMake
wants consistent performance in a section of code across compilers, it would
need a way to explicitly specify the small string size.  For example, some
are large enough to store typical target sizes - and some maybe are not.

There is also boost::container::small_vector in addition to QVarLengthArray:
http://www.boost.org/doc/libs/1_60_0/doc/html/boost/container/small_vector.h
tml

> Just run cmake (or the daemon) through a profiler and check the results.
> Doing so for the daemon (built with RelWithDebInfo) on the LLVM build dir
> and recording it with `perf --call-graph lbr` I get these hotspots when
looking
> at the results with `perf report -g graph --no-children`:
> 
> +8.67%  cmake  cmake[.]
> cmGlobalGenerator::FindGeneratorTargetImpl
> +4.21%  cmake  libc-2.22.so [.] _int_malloc
> +2.67%  cmake  cmake[.] cmCommandArgument_yylex
> +2.09%  cmake  libc-2.22.so [.] _int_free
> +2.06%  cmake  libc-2.22.so [.] __memcmp_sse4_1
> +1.84%  cmake  libc-2.22.so [.] malloc
> 
> This already shows you that you can gain a lot by reducing the number of
> allocations done. Heaptrack is a good tool for that.

Next question would be: who is calling malloc?  Or rather, what % of callers
are std::string, std::vector, other STL classes vs custom data structures?
Next question would be: what is the size of those mallocs, for each caller?

(Sorry I don't currently have an environment set up with a profiler to test
this myself at the moment.) 

> Similarly, someone should
> investigate cmGlobalGenerator::FindGeneratorTargetImpl. That does a lot of
> string comparisons to find targets from my quick glance, so indeed could
be
> sped up with a smarter string class.
> 
> But potentially you could also get a much quicker lookup by storing a hash
> map of target name to cmGeneratorTarget.

Indeed; there has got to be a way to reduce the complexity of that function
in number of targets compared, if not the low-level string comparison itself
as well.  For example, if target names are short-ish, the string class has
large enough SSO, and the underlying string class made use of vector CPU
instructions for comparison, there is probably very little to be gained
without such a hash map.  (On the other hand, if some of the previous
assumptions are not true on some common CMake platforms) 

> Seems like there's more than enough areas one could (and should) optimize
> CMake.

Indeed.  Another idea - probably unrelated to the string allocations issue,
but still - that came to mind: what if link-time code
generation/optimization is turned on?  IIRC this is not default in CMake.
Maybe CMake is sufficiently well-organized (e.g. small functions
implementations moved to header files) such that what needs to be inlined
across units, is already being inlined.  Then again, maybe it's not.  I've
seen other projects rely on this feature to keep clean organization by
keeping implementations in .CPP files without sacrificing performance, and
when you turn off LTCG performance takes a major hit... 

Also IIRC there are still a few optimizations that are turned off when CMake
is built with RelWithDebInfo instead of Release.  I forget the exact
specifics at the moment but e.g. on Visual C++ when you ask it to turn on
debug symbols, it will change the default values of some optimization flags.
So a cursory examination of the flags wouldn't reveal all cases.

However, one of my bigger performance gripes, being a primarily Windows
developer right now, is the process creation overhead, especially during
configuration.  I think that is completely dominating over any CMake code
being run 

Re: [cmake-developers] CMake daemon for user tools

[James Johnston]

> -Original Message-
> From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
> On Behalf Of Milian Wolff
> Sent: Thursday, January 21, 2016 22:31
> To: Daniel Pfeifer
> Cc: CMake Developers; Stephen Kelly
> Subject: Re: [cmake-developers] CMake daemon for user tools
> 
> > What do you think about string interning? I started a cmString class
> > [2] that stores short strings inplace and puts long strings into a
> > pool. Copies never allocate extra memory and equality comparison is
> > always O(1).
> > In addition to `operator<` (which is lexicographical) there is a O(1)
> > comparison that can be used where lexicographical ordering is not
> > required (ie. for lookup tables).
> >
> > [1] https://en.wikipedia.org/wiki/String_interning
> > [2] https://github.com/purpleKarrot/CMake/commits/string-pool
> 
> Imo, you should replace this custom code by a boost::flyweight of
std::string.
> That said, I think this can have a significant impact on the memory
footprint
> of CMake, considering how heavy it relies on strings internally. But it
also
> seems to mutate strings a lot. I've seen places e.g. where a list of
compile-
> time known identifiers is prepended with "CMAKE_" at runtime. This is slow
> with normal strings (reallocations), but will also be slow with a
flyweight or
> interning, possibly even leading to the pollution of the internal pool
with
> temporary strings.
> 
> Did you measure any impact on both, runtime speed and memory footprint
> yet?

I was wondering the same.  I would guess maybe the biggest impact would be
the inplace storage of strings for small sized strings.  But to know the
inplace buffer size would probably require some profiling and measurement of
string sizes... otherwise it is just a wild guess... 

Maybe for testing, you can swap out the string header file on your system
with one that logs allocations/string sizes, and perhaps also profiles the
time it takes to make each allocation?

The interesting question is: could inplace storage be used for 95% of the
cases such that fussing with string interning becomes unnecessary
complexity?  If so, then you mentioned equality comparison as another issue:
the interesting question there is how much time is spent on allocations vs
comparisons...

In another application I worked on, I was able to get a big improvement in
performance by replacing usage of std::vector in one place with a custom
vector that stack-allocated the first 10 items (i.e. fixed-size C array as a
member variable of the class), and then reverted to a regular vector after
that.  But to pick the number "10" required some profiling/measurement.  The
remaining use of the heap was so negligible as to not be worth improving.

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] generator expression for path slash conversion

[James Johnston]

> -Original Message-
> From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
> On Behalf Of Kislinskiy, Stefan
> Sent: Thursday, September 24, 2015 15:09
> To: Brad King
> Cc: cmake-developers@cmake.org
> Subject: Re: [cmake-developers] generator expression for path slash
> conversion
> 
> Great, thank you, Brad & David.
> 
> Regarding the ExternalProjectShellPathGenex test: I wrongly assumed that
> the WIN32 variable wouldn't be set when using MSYS and the like. Would it
> make sense to keep the test when changing the WIN32 check to MSVC?

Just an FYI:  MSVC can still be set; it's an indicator of the *compiler* in
use, not the *generator*.  For example, it could be set if using cl.exe with
Ninja.

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [PATCH] [CMake 0015674]: Windows: Correctly determine Windows version

[James Johnston]

> -Original Message-
> From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
> On Behalf Of Brad King
> Sent: Friday, September 18, 2015 18:24
> To: Gilles Khouzam
> Cc: cmake-developers@cmake.org
> Subject: Re: [cmake-developers] [PATCH] [CMake 0015674]: Windows:
> Correctly determine Windows version
> 
> On 09/10/2015 07:24 PM, Gilles Khouzam wrote:
> > This patch adds a simple version manifest
> > Source\cmake.version.manifest to the CMake executables.
> 
> After working out the support for manifests across all generators as
> discussed elsewhere in this thread, I've added your manifest file to
CMake's
> own build:
> 
>  Windows: Fix CMAKE_HOST_SYSTEM_VERSION on Windows >= 8 (#15674)
>  http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cdd15551

While on the subject, there are some other additions to the manifest you
might want to consider.









The above block will disable UAC heuristics on Windows Vista and above, such
that elevation won't ever be requested.  It will also disable the Program
Compatibility Assistant for Vista IIRC.  (as I understand it, supportedOS
wasn't added until Windows 7.  Why there is a GUID in there for Vista, I'm
not sure.  App manifests are not the best documented thing on MSDN IMHO.)
It would probably be appropriate to use this for all CMake executables.







The above block will declare a preference to use the newer common controls
library introduced with Windows XP.  Without this, Windows will fall back on
the old Windows 2000 era of common controls (i.e. Windows 95 style buttons).
This would be appropriate for cmake-gui.  Since it uses Qt though, the
effect is probably minimal, but if some API were called such that native
Win32 controls were used somewhere, this would improve the appearance.



http://schemas.microsoft.com/SMI/2005/WindowsSettings;>
True/PM




The above block will make the cmake-gui DPI aware, such that DPI
virtualization is disabled on Windows Vista and above.  In laymen's terms,
it fixes "blurry text".  It looks like Qt already does this (I just tested
CMake 3.3 on a high DPI profile.)  See http://doc.qt.io/qt-5/highdpi.html.
Probably they do it by calling the SetProcessDpiAwareness function.
However, this API has some caveats; from MSDN:

" The SetProcessDPIAware function in Windows Vista and later versions sets
the current process as DPI-aware. However, the use of the SetProcessDPIAware
function is discouraged. For example, if a DLL caches DPI settings during
initialization, invoking SetProcessDPIAware in your application might
generate a possible race condition. For this reason, we recommend that an
application enable DPI awareness by using the application's assembly
manifest rather than by calling SetProcessDPIAware." ---
https://msdn.microsoft.com/en-us/library/windows/desktop/dn469266(v=vs.85).a
spx

And also:

" If you use the SetProcessDpiAwareness method to set the DPI awareness
level, you must call SetProcessDpiAwareness prior to any Win32API call that
forces the system to begin virtualization."

The safer path seems to me to be put it in the manifest, as recommended by
MSFT.  Note there are two possible values:  "True" and "True/PM".  "PM"
stands for per-monitor DPI aware (introduced with Windows 8.1).  Looks like
Qt supports PM now, but since it was introduced in Windows 8.1, probably
only very recent versions of Qt.  You'd have to be careful what is in the
manifest agrees with the version of Qt being used. (e.g. you don't want to
declare "True/PM" on a non-PM older version of Qt, else no per-monitor
scaling would happen at all - virtualized or otherwise).

Since Qt does call SetProcessDPIAware and cmake-gui does seem to scale OK
right now, including this block may end up being a little "pointless" in
practice, despite the caveats from MSDN above.



Finally there is this block; I'm not exactly sure which versions of Windows
might actually require this or what.  I'm a little fuzzy on why I included
it.  Most of the built-in Windows accessories seem to have this in their
manifests though.  IIRC it's especially needed with the common controls
manifest block.

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CMake user-provided manifest files

[James Johnston]

> -Original Message-
> From: Brad King [mailto:brad.k...@kitware.com]
> Sent: Wednesday, September 16, 2015 20:13
> To: James Johnston; 'Gilles Khouzam'
> Cc: cmake-developers@cmake.org
> Subject: Re: [cmake-developers] CMake user-provided manifest files
> 
> On 09/16/2015 03:43 PM, James Johnston wrote:
> > So the problem is the current directory when linking is going to be
> > root CMAKE_BINARY_DIR, and the relative path "DPIAware.manifest" will
> > attempt to be used, which of course isn't there as it needs to look in
> > "subdir\DPIAware.manifest".
> 
> This should fix that:
> 
>  fixup! Add support for *.manifest source files with MSVC tools
>  http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e78b5408
> 
> I'll also work on integrating your test case.
> 
> Thanks,
> -Brad

That fixed it; just tested building a pile of projects with both Ninja and
VS2008 generators (with VS2008 used with Ninja). 

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CMake user-provided manifest files

[James Johnston]

> -Original Message-
> From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
> On Behalf Of Brad King
> Sent: Monday, September 14, 2015 20:16
> To: James Johnston
> Cc: cmake-developers@cmake.org
> Subject: Re: [cmake-developers] CMake user-provided manifest files
> 
> On 09/11/2015 11:45 AM, James Johnston wrote:
> > Because CMake already does *some* things with the linker it makes it
> > impossible for me to cleanly specify and use link.exe manifest-related
> > switches myself.  For example, it hard-codes usage of link.exe
> > /MANIFESTFILE and the user can't specify their own location.  And the
> > locations it does use are undocumented.
> 
> Is there a use case for doing something with the linker-generated manifest
> file other than passing it to a "mt" invocation along with (possibly)
user-
> specified manifest files?  If not then there is no reason to make the
manifest
> file location public.

I can't think of anything like that.  The only reason I was using the
undocumented linker-generated manifest location was to merge it with my own
manifest using mt.exe.  So I agree, I can't think of a reason the location
needs to be publicly documented/available if CMake can cleanly handle
user-provided manifests to merge.
 
Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [PATCH] [CMake 0015674]: Windows: Correctly determine Windows version

[James Johnston]

Well this is quite cool. :)  It's been an issue I've wanted to see addressed
as well but haven't had the time to look at it.  Not because of CMake itself
not having a manifest (the improper GetVersionEx detection didn't affect
me), but because I need to embed an app manifest in my own programs.

I have some ideas while you are on the subject:

1.  No documentation files were updated.  I wonder if there is a good place
in the docs for this?
2.  If the user doesn't use the Visual Studio generator, still support
embedding the manifest.  For example, if I use Ninja generator in
conjunction with Visual C++'s link.exe I'd like user-specified manifests to
be merged with mt.exe.

The second request is currently a big pain point with CMake to get working
correctly.  Ideally I want to reproduce the VS way of doing things, which
is:

a.  Have link.exe generate default manifest for referencing the CRT
side-by-side assemblies.
b.  Use mt.exe to put it all together in the output.

Because CMake already does *some* things with the linker it makes it
impossible for me to cleanly specify and use link.exe manifest-related
switches myself.  For example, it hard-codes usage of link.exe /MANIFESTFILE
and the user can't specify their own location.  And the locations it does
use are undocumented.  See Source/cmcmd.cxx and search for vs_link_exe to
see what I'm talking about.  (For that matter, git grep CMake for
vs_link_exe and vs_link_dll to see how those commands end up getting used in
the first place).

Ultimately for now I wound up with this kludge to get link.exe intermediate
manifest AND my own:

if(CMAKE_CONFIGURATION_TYPES) # multiple-configuration
generator?
# WARNING:  This magic directory location is an undocumented
aspect of the
# Visual Studio generator.
list(APPEND inputManifests
 
${CMAKE_CURRENT_BINARY_DIR}/${AAM_TARGET}.dir/$/$<TARGET_FILE_NAME:$
{AAM_TARGET}>.intermediate.manifest
)
else()
# Can't change /MANIFESTFILE because CMake assumes the
default... (see above)
list(APPEND inputManifests
$<TARGET_FILE:${AAM_TARGET}>.manifest)
endif()
list(APPEND inputManifests my_own_manifests.manifest)

Which is then fed into add_custom_command( POST_BUILD COMMAND mt.exe
-manifest ${inputManifests} ).  Basically I'm hard-coding the
hard-coded /MANIFESTFILE: locations used by various CMake generators, since
it doesn't work for me to try to change it.  Obviously the situation is not
ideal because CMake makes no promises about where it puts the intermediate
manifests from the linker.

It would be certainly be nice if CMake supported user-provided manifests
across most/all Windows generators as first-class support, not just Visual
Studio 10.  Especially the make-like tools (various Makefile generators,
Ninja).  :)

Best regards,

James Johnston

From: cmake-developers [mailto:cmake-developers-boun...@cmake.org] On Behalf
Of Gilles Khouzam
Sent: Thursday, September 10, 2015 23:25
To: Brad King
Cc: cmake-developers@cmake.org
Subject: [cmake-developers] [PATCH] [CMake 0015674]: Windows: Correctly
determine Windows version

This patch fixes the issue where on Windows 8 and above, by default the
system version returned is always Windows 8.

More info on this can be found on
http://blogs.msdn.com/b/cjacks/archive/2009/03/27/manifesting-for-compatibil
ity-on-windows-7.aspx

In order for GetVersionEx to work properly, a manifest must be embedded in
the exe telling it to ignore the new behavior and give the proper version.

This patch adds support to specify manifests and teaches CMake how to embed
them using Visual Studio.
This patch adds a simple version manifest Source\cmake.version.manifest to
the CMake executables.
This removes the use of RtlGetVersion and properly uses GetVersionEx which
will now return the proper operating system version for all operating
systems until Windows 10.
This patch also fixes a potential buffer overrun and improper call of
GetVersionEx in SystemTools.cxx where GetVersionExW is called with a
OSVERSIONINFOEXA struct causing a failure since the size is improper for the
API. Fixing this, GetVersionEx should never fail anymore.
Update the OSVersionAndName for new versions of Windows released after
Windows 7.

This change will require a double compile of CMake (I'm not sure if this is
what the dashboards are doing) to have the functionality enabled, since the
first step will build a new CMake that will know how to embed a manifest.
The second compile with the version of CMake that knows how to embed a
manifest, will then embed the manifest in CMake enabling this feature.

Thanks

~Gilles

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offe

Re: [cmake-developers] [CMake] Visual Studio - Ninja Generator

[James Johnston]

> -Original Message-
> From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
> On Behalf Of Brad King
> Sent: Wednesday, September 09, 2015 15:00
> To: cmake-developers@cmake.org
> Subject: Re: [cmake-developers] [CMake] Visual Studio   -
> Ninja Generator
> 
> On 09/02/2015 03:34 PM, James Johnston wrote:
> > useful if the Visual Studio generators in CMake were refactored
> > somewhat
> 
> Even without the C# motivation I think factoring out a "MSBuild" generator
> infrastructure internally will be useful.  Currently we call it "VS 10"
> because it happened to be the first version to use MSBuild project files.
> Are you proposing to have some kind of internal object model for the
> MSBuild project files to separate their construction from the actual
> generation?

Right, something like that.

Another motivation: if somebody ever wanted to write a generator for
Embarcadero C++ Builder projects... guess what format the project files are
in... MSBuild...  this otherwise has nothing to do with VStudio...

Again, not something I'm working on right now but just putting some ideas
out there. :)

> 
> > it would be useful to have Visual Studio available as an "Extra" CMake
> > generator.  For example, specification of "Visual Studio 2015 - Ninja"
> 
> This functionality sounds reasonable but the name of the extra/generator
> pair looks funny when spelled out that way.  We should consider having
> another way to specify the extra generator.

This name is consistent with the other extra generators:
http://www.cmake.org/cmake/help/v3.3/manual/cmake-generators.7.html#id12

"[Extra] generator names have the form  -
."

But I agree it is just confusing at this point.  Maybe this format could be
deprecated in favor of a new cmake.exe switch to specify an extra generator.

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] Visual Studio - Ninja Generator

[James Johnston]

h C# project - even when using 
non-VS generators like Ninja.  The Ninja build rule for a C# project would just 
be to invoke MSBuild.exe and build that one project.  (Only generate projects, 
not solutions.)  But this requires the ability for a CMake language to invoke 
the MSBuild-generating code inside CMake to spit out a Visual Studio project 
for each C# target.  Benefit: now CMake doesn't reimplement undocumented 
behavior of Microsoft's MSBuild targets; we just use them directly as a black 
box.

Best regards,

James Johnston

From: CMake [mailto:cmake-boun...@cmake.org] On Behalf Of Guillaume Dumont
Sent: Monday, August 31, 2015 16:49
To: Gonzalo
Cc: cm...@cmake.org
Subject: Re: [CMake] Visual Studio   - Ninja Generator

@Hendrik Sattler
I have not experimented with the /MP flag that much, but this won't solve my 
problem when the project contains a lot of CUDA files. The /MP flag as not 
effect there. 
Yes I could indeed create a custom build target and create additional build 
trees to build using ninja in there but this is suboptimal in my opinion.

@Gonzalo
Yes this is precisely what I do but then you only get the ninja build files. No 
solution for debugging etc.
My question is more about the difficulty of creating a new generator that makes 
the use of ninja as transparent as possible to the user instead of writing 
custom CMake code to do this in my own projects.

On Mon, Aug 31, 2015 at 12:29 PM, Gonzalo <ggarr...@gmail.com> wrote:
s it has changed names a couple of times).  That shoul




-- 
Guillaume Dumont
=
dumont.guilla...@gmail.com

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] ExternalProject: Use native paths as substitute for directory tokens

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of Kislinskiy, Stefan
 Sent: Thursday, August 27, 2015 13:44
 To: Brad King; CHEVRIER, Marc; David Cole
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] ExternalProject: Use native paths as
 substitute for directory tokens
 
 This is pretty nice from the perspective of an engineer. But I wonder if
it
 wouldn't be kind of over-engineered in the sense that it would be a rather
 hidden feature for the purpose? -- I probably wouldn't think of searching
for
 a generator expression in the documentation when I have trouble with the
 replacement of directory tokens. Then again, there could be a hint in the
 ExternalProject documentation which would be fine I guess. One way or
 another, I would be happy if we could determine what the patch should
 provide exactly so that we come to an end. :-) Thank you for all the
feedback
 so far.

I think Brad raises an excellent point here though I hadn't thought of...
implementing the underlying functionality as a generator expression allows
the problem to be solved outside of the scope of ExternalProject when it
arises...  For example if I add_custom_command as a custom build step and
need to pass a native path argument to it - a path known only at generate
time.  I guess it is not possible right now?

 Another approach is to introduce a generator expression to transform the
 path slash style.  Then it could be used both for ExternalProject and in
other
 custom commands.  E.g.
 
  $PATH_FOR_SHELL:SOURCE_DIR/bootstrap${shell_ext}
 
 In this case SOURCE_DIR would be replaced with forward slashes by
 ExternalProject but then at generate time CMake would evaluate the genex
 to convert slashes as needed.

I would vote naming it TO_NATIVE_PATH instead of PATH_FOR_SHELL, for
consistency with the existing parameter in the file command.  There could
then also be a corresponding TO_CMAKE_PATH gen-exp, although I wonder if it
would be used much.  (Then again, do people even use file(TO_CMAKE_PATH)
very much?) 

James


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] ExternalProject: Use native paths as substitute for directory tokens

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of Kislinskiy, Stefan
 Sent: Wednesday, August 26, 2015 10:03
 To: CHEVRIER, Marc; David Cole
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] ExternalProject: Use native paths as
 substitute for directory tokens
 
 I see. Added native-style path replacements for SOURCE_NATIVE_DIR and so
 on. Also extended the documentation accordingly.

Some relatively major/minor/nitpicky thoughts in the interests of
thoroughness:

 * There's no test code.  It would be good if there was some code to
explicitly test these options for nightly testing.  For that matter, I think
the original tokens weren't thoroughly tested either (e.g. a quick grep for
TMP_DIR doesn't show usage of that token at all in the tests).  The whole
situation could therefore use some help, I think.  There are several
existing tests of ExternalProject that could probably be extended to test
these tokens as needed.

 * Token replacement is done for WORKING_DIRECTORY and BYPRODUCTS parameters
of add_custom_command.  You'd best be sure those parameters of
add_custom_command can handle a native path, or else not convert to native
paths - even for *_NATIVE tokens - when the path is destined for one of
those parameters.  (I'd tend to assume the command *could* handle it
properly since a lot of users might inadvertently pass them, but it's
something that should be double-checked.  If the command is passing those
paths as-is to the generator, correct operation could even vary by
generator.)  The easy way out might be to have a parameter on
_ep_replace_location_tags declaring whether to ACTUALLY convert native path
tokens to native paths, and then callers to that macro can decide if they
can / should handle native paths.

 * It would be useful if the documentation explicitly stated that the old
BINARY_DIR  friends variables are using CMake paths; currently the
documentation does not say - which was a shortcoming of the original docs
(as you found out accidentally, it is CMake paths).

 * Nothing explicitly to do with your patch, but the documentation doesn't
state that any of the tokens are valid in ExternalProject_Add; it may be
useful to audit the options of ExternalProject_Add and see where they can be
used, and document accordingly.  (Some are obvious, like CONFIGURE_COMMAND,
but some are less obvious, like CMAKE_CACHE_ARGS / CMAKE_CACHE_DEFAULT_ARGS
which appear to have their own special handling for these tokens, yet none
of that is documented.  Even a newcomer might not realize the obvious, if
they don't realize that ExternalProject_Add is implemented via
ExternalProject_AddStep.)

 * New features should also be documented in Help/release/dev so that it
makes it into the release notes of the next CMake version (i.e. so people
notice your new feature).

I leave it to others to decide if adding the new *_NATIVE tokens is the
right way, or if there's a better way - as I myself am a newcomer to CMake.
I think I like it though. :)

Best regards,

James Johnston 



-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] ExternalProject: Use native paths as substitute for directory tokens

[James Johnston]

Hi,

Funny you are mailing the list about this, since I just ran into this same 
issue today building something totally different from Boost...  In this case 
it's a build tool that thinks the /U in C:/Users is a new command line 
argument, that isn't recognized - and then the subsequent s also ends up 
unrecognized... and it all fails...  And it has nothing to do with the working 
directory, so _Add_Step(WORKING_DIRECTORY) isn't a possible workaround for me.

I think the issue with globally making this change to the existing tokens is 
that there could be some external tool/program that is EXPECTING to get CMake 
paths, not native paths.  Who knows?  I am guessing that is what David Cole was 
concerned about.

Maybe the right answer is to introduce some NEW tokens while leaving the 
behavior of the old ones unchanged.  E.g. BINARY_DIR_NATIVE etc.  It would be 
good if the patch updates the documentation of ExternalProject and clearly 
states the path format of BINARY_DIR vs BINARY_DIR_NATIVE.  Then the user 
can pick whichever one suits them best, depending on the tool being invoked.

Furthermore, sometimes BINARY_DIR_NATIVE still needs to be replaced with a 
CMake path, not native path.  For example, if the token is being found in a 
property like WORKING_DIRECTORY that eventually gets passed to 
add_custom_command(WORKING_DIRECTORY) then I'm guessing still has to be a CMake 
path.  I am guessing this is what David Cole was also concerned about.

I still think your original method of building Boost is a bit unusual and would 
be better served by _Add_Step with a custom working directory - because that's 
the publicly documented/standard way of changing the working directory, but 
that is up to you.  :)

Best regards,

James Johnston


 On Thu, 20 Aug 2015 14:37:08 +  Stefan Kislinskiy 
s.kislins...@dkfz-heidelberg.de wrote  

  Hi, 
   
  thank you for our suggestions. I am aware that I can solve my example 
  differently and that it might look not directly connected the proposal, but 
  well, it is an example just to show a single case and why it matters. :) I 
  did not want to discuss the example itself. Working around here would just 
  resolve a symptom. 
   
  My point is the overall problem that would persist: A big part of 
  ExternalProject is to issue commands for predefined and custom steps. Those 
  commands are supposed to be executed by the shell/command line. According to 
  the documentation and the source code of ExternalProject, directory tokens 
  are mainly supposed to be replaced in commands. It is my understanding, that 
  it is a bug, if CMake isn't able to assemble these commands correctly. This 
  would include usage of the correct path style of the OS for shell/command 
  line commands. As directory tokens are replaced internally right before a 
  shell/command line command is assembled, I can't see why this would be kind 
  of API-breaking. You cannot interfere in your CMake code with these 
  internal replacements. 
   
  Therefore I would still prefer my solution as it is pretty simple without 
  adding even more features to ExternalProject and in my opinion without 
  breaking code in the wild. It is a true bug fix instead of a feature request 
  for working directories, which is a different topic that just coincidentally 
  arised because of my specific example I guess. The features you described 
  wouldn't fix the actual bug. 
   
  As you were not sure if my approach would even fix my problems: It does of 
  course and this is what I am currently doing and what I tested extensively 
  before creating the patch. :) Regarding your quote from the 
  add_custom_command documentation I can tell you that this is how things are 
  currently done in ExternalProject and always were as far as I know, for 
  example (from ExternalProject.cmake): 
   
add_custom_command( 
  OUTPUT ${stamp_file} 
  BYPRODUCTS ${byproducts} 
  COMMENT ${comment} 
  COMMAND ${command} 
  COMMAND ${touch} 
  DEPENDS ${depends} 
  WORKING_DIRECTORY ${work_dir} 
  VERBATIM 
  ) 
   
  Best regards, 
  Stefan 
   
   
  -Original Message- 
  From: cmake-developers [mailto:cmake-developers-boun...@cmake.org] On Behalf 
  Of James Johnston 
  Sent: Donnerstag, 20. August 2015 15:37 
  To: cmake-developers@cmake.org 
  Subject: Re: [cmake-developers] ExternalProject: Use native paths as 
  substitute for directory tokens 
   
   -Original Message- 
   From: cmake-developers [mailto:cmake-developers-boun...@cmake.org] 
   On Behalf Of Kislinskiy, Stefan 
   Sent: Thursday, August 20, 2015 09:02 
   To: David Cole 
   Cc: cmake-developers@cmake.org 
   Subject: Re: [cmake-developers] ExternalProject: Use native paths as  
   substitute for directory tokens 

   Hi David, 

   Example excerpt (it is not possible to change the working directory  
   for 
  the 
   CONFIGURE_COMMAND as it is fixed to the BUILD_DIR, which might

Re: [cmake-developers] ExternalProject: Use native paths as substitute for directory tokens

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of Kislinskiy, Stefan
 Sent: Thursday, August 20, 2015 09:02
 To: David Cole
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] ExternalProject: Use native paths as
 substitute for directory tokens
 
 Hi David,
 
 Example excerpt (it is not possible to change the working directory for
the
 CONFIGURE_COMMAND as it is fixed to the BUILD_DIR, which might not be
 sufficient):

This doesn't really directly have to do with your proposal, but what if an
option was added to change the working dir of the CONFIGURE_COMMAND?  E.g.
WORKING_DIRECTORY_CONFIGURE.  And suppose you'd have it recognize the
various tags like SOURCE_DIR, etc.  This might be useful to add to other
steps as well, and would be more portable than your solution which is using
cmd.exe-specific commands.  You'd want to audit for any resulting breakage
(e.g. does ExternalProject make assumptions that the working directory of
CONFIGURE is always the binary dir? - e.g. a relative path being used
somewhere.  And probably only allow specification of
WORKING_DIRECTORY_CONFIGURE if a CONFIGURE_COMMAND was also specified, as
the built-in commands certainly assume the default working dir.)

In your situation though, I'm not sure it's strictly needed.  From your
sample, it looks like you're building boost.  In your case what if you:

 * Use ExternalProject_Add_Step to bootstrap.  You can specify a
WORKING_DIRECTORY here.  Note one problem: you can't do out of source build
of b2, which breaks user expectations.
 * Then use ExternalProject_Add_Step to build Boost.

Yes, using _Add_Step is somewhat of a workaround, but in this case, I've
found it wasn't much of a burden at all.  In fact the only case I can think
of where it WOULD be a burden would be if the configure step is CMake.  But
then you wouldn't need to change the working directory; changing it would
break CMake.  In practice nobody will want to change WORKING_DIRECTORY
unless it's a custom command and then it's easy to use _Add_Step anyway.
That said, it might still be considered a little undesired and so maybe my
proposal above would be a better way to handle it.

Corrections from maintainers and others on the above commentary are
welcome...

 
 set(bootstrap_cmd SOURCE_DIR/bootstrap${shell_ext}
 ${bootstrap_toolset})
 
 if(WIN32)
   set(bootstrap_cmd pushd SOURCE_DIR COMMAND ${bootstrap_cmd}
 COMMAND popd)
 endif()
 
 ExternalProject_Add(Boost
   ...
   CONFIGURE_COMMAND ${bootstrap_cmd}
   ...
 )

From add_custom_command:  If more than one COMMAND is specified they will
be executed in order, but not necessarily composed into a stateful shell or
batch script.

So I am not sure your approach will work for you even if you fix the issue
with path slashes.

James


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] ExternalProject: Use native paths as substitute for directory tokens

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of David Cole via cmake-developers
 Sent: Thursday, August 20, 2015 21:21
 To: James Johnston
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] ExternalProject: Use native paths as
 substitute for directory tokens
 
 It's exactly what I am concerned about:
 
 You're asking to change the behavior of something for EVERYONE to solve a
 problem which you have encountered. If you change it the way you have
 proposed, you will cause problems for others. It has worked the way it is now
 since ExternalProject_Add was introduced in CMake 2.8. Changing it
 unconditionally the way you propose is simply not feasible for backwards
 compatibility.
 
 I think commands that take native paths ought NOT to use the *_DIR
 replacement values, and instead, ought to pass in variables that contain the
 native paths in the first place.

Right, agreed that the original *_DIR behavior would need to remain 
unchanged.  But how would you know what the binary directory of the external 
project is, before calling ExternalProject_Add?  It's too early to call 
ExternalProject_Get_Property(target BINARY_DIR).  I assume that is why 
BINARY_DIR and friends exist, as the location will be affected via various 
ExternalProject parameters.  From the doc:

The *_DIR options specify directories for the project, with default 
directories computed as follows snip

Quite a set of rules follow and if one is trying to use the binary directory in 
the path to some build tool, it's handy to put BINARY_DIR in the command.

But if you need a native path for your build tool, BINARY_DIR doesn't work as 
Stefan Kislinskiy noted, which is why I suggested making new tokens might be a 
viable alternative, e.g. BINARY_DIR_NATIVE?

Best regards,

James Johnston 

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] Clarification requested on #10126 (CMake creates files with wrong permissions)

[James Johnston]

Hi,

I figured I'd take a look at resolving #10126:
http://www.cmake.org/Bug/view.php?id=10126

The fix proposed by Brad King at this ticket involves two parts.  I'd like
to seek clarification on this ticket.

1.  It is proposed to add new file permissions to INSTALL and FILE commands
called READ/WRITE/EXECUTE.

a.  So to clarify, you'd want existing OWNER_* / GROUP_* / EXECUTE_*
options to remain unmodified?  i.e. if you install(PERMISSIONS GROUP_WRITE)
this will still NOT respect the umask, correct?  (i.e. the only way to
respect the umask would be to use the NEW READ/WRITE/EXECUTE options?)

b.  If the answer to (a) is yes, what do you want to do if the user
specifies install(PERMISSIONS WRITE GROUP_WRITE)?  Is that an error?  Should
we just have the WRITE take precedence and respect the umask anyway, even
though GROUP_WRITE was specified to not respect umask?  (If the answer to
(a) is no, then obviously there's no harm in specify WRITE GROUP_WRITE
because it's purely redundant.)

c.  Would it be better to introduce a new permissions keyword for
respecting the umask for the entire set of keywords?  E.g.
install(PERMISSIONS RESPECT_UMASK GROUP_WRITE).  This could be done in
conjunction with new READ/WRITE/EXECUTE, so for example:
install(PERMISSIONS RESPECT_UMASK WRITE) would specify owner/group/world
write while respecting umask, and install(PERMISSIONS WRITE) would do the
same but NOT respect the umask.  What I like about this: (i) preserves
compatibility as if you answer yes to (a) above, (ii) avoids the issues
from (b) above.

d.  If the answer to (c) is yes, should that keyword become the default,
subject to a new policy?  (And introduction of a different NO_RESPECT_UMASK
keyword).

2.  Proposal to modify default permission copying of configure_file and
friends: I may have questions here but haven't thought it through yet.

Finally, to resolve the issue as reported by the original submitter:

Fix works almost perfectly, yet there are some files still having 0644
rights:
in builddir/CMakeFiles/ it's
- CMakeCCompiler.cmake
- CMakeCXXCompiler.cmake
- CMakeSystem.cmake

3.  Won't the proposal from #2 above fix this problem?  As in, #1 then
because a nice-to-have thing but not necessary to fix the original problem
with CMake?  (Or do we need to go on and modify the INSTALL commands in
CMake itself to use the new #1 feature above to change CMake's
installation?)

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Setting up environment using ExternalProject_Add

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of David Cole via cmake-developers
 Sent: Wednesday, August 12, 2015 15:53
 To: Brad King
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] Setting up environment using
 ExternalProject_Add
 
 One thing to consider before embarking on modifying CMake for required
 environment purposes is that it might be just as simple to force (or at
least
 strongly encourage) your project developers to use the proper environment
 for running the whole build.
 
 You could, for example, have an initial external project which all
others
 depend on which verifies the proper environment is set when it is called.
If
 not, it fails with an error, and prevents downstream bits from building.
 
 snip

I think your situation is simpler than mine - i.e. if I'm reading this
right, all your ExternalProjects are built for the same Visual C++ version
and CPU architecture - such that the user can/should run VCVarsAll before
invoking the CMake superbuild?

What I'm trying to do:  A CMake superbuild that expects NO environment set
up, and NO languages configured.  It:
 * Builds Visual C++ 32-bit
 * Builds Visual C++ 64-bit
 * Builds some other non-Visual-C++ compilers/architectures

All within the same superbuild.  This is due to the fact that CMake doesn't
support multiple architectures directly (and probably won't for a long time,
if ever).  So, each ExternalProject_Add needs to have its own unique
environment set up.  I don't think it makes sense to try to run
VCVarsAll.bat before invoking the superbuild - in fact, doing so is
detrimental (i.e. increases risk of building for the wrong CPU architecture
if a particular VCVarsAll.bat invocation silently malfunctions).

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Setting up environment using ExternalProject_Add

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of Brad King
 Sent: Wednesday, August 12, 2015 14:10
 To: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] Setting up environment using
 ExternalProject_Add
 
 On 08/11/2015 10:41 AM, James Johnston wrote:
  1.  CMake creates a temporary shell script / batch file (the shell
  chosen based on the platform CMake compiled for - cmd.exe on Windows,
 sh on POSIX).
 
 I don't think CMake needs to be the one to generate this.  The complexity
of
 the file-based or command-line-based interfaces you discussed shows that
it
 would be tricky to have CMake generate it.

 snip

OK...

  enhancing e.g. add_custom_command as I proposed might make it easier,
  as this command could be aware of compiled EXE targets.
 
 With file(GENERATE) the project code can easily create scripts that
reference
 executable targets and such.  One would just need to tackle this feature
 request first to get file permissions right:
 
  http://www.cmake.org/Bug/view.php?id=15653

Indeed; that looks like a useful feature to have to begin with, and a
requirement if ExternalProject is to generate its own script...

 
 Therefore I do not think add_custom_command needs any hook for this.
 One would simply give the script as the COMMAND.

Right, obviously not needed if no cmake -E option is added/enhanced and if
we want ExternalProject_Add (and anyone else needing custom targets) to make
their own script...

 
  we'd want to add features to ExternalProject that use the new CMake
  functionality to finally solve the original problem.
 
 Yes, this part certainly needs an update to ExternalProject.  Currently
when
 one specifies an individual step_COMMAND like CONFIGURE_COMMAND,
 then one could simply give it the environment-set-and-launch script
directly.
 What ExternalProject needs is an interface to specify the environment-
 setting script for use around otherwise default-generated commands for
 each step.  For example, a step_ENVIRONMENT setting (which is perhaps
 not allowed in combination with step_COMMAND).
 This approach is similar to your original alternative (3).

I'll explore that option some more, after the feature you linked to is
addressed.   one could simply give it the environment-set-and-launch script
directly ... This script is obviously platform-specific, yet still needs
knowledge of the command that ExternalProject will generate to launch the
sub-CMake - which the user of ExternalProject obviously does not know.  I am
not sure that introducing some shell-specific knowledge to ExternalProject
can be avoided in this case, so hopefully it will not be a problem to do
this.

There are two ways I immediately think of - trying to clarify what I think
you are suggesting:

1.  The new step_ENVIRONMENT setting points to a environment-set script
that does not try to do any launching (e.g. you could pass VCVarsAll.bat
directly as the step_ENVIRONMENT).  ExternalProject determines which shell
to generate for, and generates the environment-set-and-launch script
itself from scratch (like how I originally proposed with the cmake -E
enhancement).

In other words, ExternalProject(snip CONFIGURE_ENVIRONMENT
$ENV{VS100COMNTOOLS}/../../VC/vcvarsall.bat x86 CMAKE_GENERATOR Ninja
CMAKE_CACHE_ARGS snip) and away the user goes..

2.  The new step_ENVIRONMENT setting points to an
environment-set-and-launch script that is expected to be configured with
configure_file by ExternalProject.  So e.g. the user provides a batch file
that calls VCVarsAll.bat, and then there is a magical
${EXTERNAL_PROJECT_COMMAND} or whatever that the user is expected to provide
at the end of the template batch file that they give to ExternalProject.  I
suspect this will simplify ExternalProject a little bit at the cost of more
boiler-plate code that the user has to provide (e.g. can't directly call
VCVarsAll.bat in the common case, must manage their own scripts).  And gives
the user opportunity to really mess up ExternalProject operation (e.g.
forgetting to include ${EXTERNAL_PROJECT_COMMAND}, or forgetting to check
ERROR_LEVEL in a shell-specific way after calling EXTERNAL_PROJECT_COMMAND,
etc.).  I think this would not be as user friendly, but perhaps slightly
easier for ExternalProject to handle.  But at the end, the configure_file
substitution for ${EXTERNAL_PROJECT_COMMAND} is still potentially
shell-specific

In other words, ExternalProject(snip CONFIGURE_ENVIRONMENT
ProjectEnvironment.bat.in CMAKE_GENERATOR Ninja CMAKE_CACHE_ARGS snip)

And ProjectEnvironment.bat.in:

@echo off
call %VS100COMNTOOLS%/../../VC/vcvarsall.bat
rem Users better not forget this next line...
rem ExternalProject would configure_file this.
${EXTERNAL_PROJECT_COMMAND}
rem If a user adds more commands to the end of the batch file, they'd need
this:
if ERRORLEVEL 1 exit /b %ERRORLEVEL%

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic

Re: [cmake-developers] Setting up environment using ExternalProject_Add

[James Johnston]

 -Original Message-
 From: Brad King [mailto:brad.k...@kitware.com]
 Sent: Tuesday, August 11, 2015 13:35
 To: James Johnston
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] Setting up environment using
 ExternalProject_Add
 
 On 08/11/2015 12:49 AM, James Johnston wrote:
  CONFIGURE_ENVIRONMENT_COMMAND path to VCVarsAll.bat
 [snip]
  add_custom_command(snip ENVIRONMENT_COMMAND path to
 vcvarsall.bat
  COMMAND ${CMAKE_PROGRAM} -GNinja snip)
 
 A problem with both of these approaches is that on UNIX and in Make the
 environment does not persist after the command exits.

Exactly right, I'm aware... and also it seems there is no
good/easy/non-hackish way to capture the environment from a child process
after it terminates, either - that I'm aware of.  (It would be nice if we
could run a process, then capture its environment after it exits, but I
don't think it can be done.)

 Whatever wrapper
 tool is used must set the environment and also launch the real command to
 be run under that environment.  This is what the env tool (and recently
 cmake -E env) does.

Right, but the problem with cmake -E env is it assumes you know what
variables you want to set - we don't.  Only a shell script / batch file
knows in the case of VCVarsAll.bat...

I think I mentioned this solution in the original e-mail but in more detail,
what I'm proposing with the proposed cmake -E run_commands or whatever we
want to call it would be the following.  In fact upon further thought, it
might be better to see about enhancing the cmake -E env command but I am
not sure if this is possible due to the syntax of cmake -E env.  Anyhow
the basic idea is this: 

1.  CMake creates a temporary shell script / batch file (the shell chosen
based on the platform CMake compiled for - cmd.exe on Windows, sh on POSIX).
2.  CMake calls one or more environment setup commands.  These are pulled
in using call / source.  Users would be required to provide scripts for
environment setup in the format of the native shell (e.g. you can't call a
bash script from a Windows .bat file).  For example:

@echo off
rem User script provided to CMake, including some arguments to the
script
call dir\VCVarsAll.bat x86
rem An additional environment variable explicitly specified by the user
set MyVar=Value
cmake -GNinja .

Or:

#!/bin/sh
source setMyEnv.sh
cmake -GNinja .

Error handling has been cut for brevity, but we would want the script to
check for ERRORLEVEL / $? after each line above.  As you can see, the
problem of environment not being retained has been solved by using call
and source.

3.  Then CMake runs the above script as normal and checks the exit code as
normal.

It's basically a close cousin of what cmake -E env does in that we: (1) do
some things to set up the environment, (2) run the specified EXE.  Ideally
cmake -E env could be enhanced with the new functionality, but since we now
want to pass an arbitrary list of commands, each with potentially unlimited
parameters of any format, I'm not sure how that should look in practice.  It
might be easier to make a new command cmake -E env2 or whatever (need to
find a better name):
  * It could be passed a file.  The file contents would have a series of
lines, either SET / UNSET name=[value], SOURCE VCVarsAll.bat x86, or
COMMAND cmake.exe snip.  Newlines delimit commands.
  * It could be passed all the commands directly on the command line.  A
recognized -- switch delimits commands.  E.g. cmake -E env2 --unset NAME
--set NAME=VALUE --source VCVarsAll.bat x86 --command cmake.exe snip
--source moreSetup.bat --command somethingelse.exe
Downsides with first method: file adds complexity.  Downsides with second
method:  subject to command line length limits, though probably not an issue
in practice.  Also, what if user wants to pass --unset or --set
parameter to their command as an argument?  Without a means of escaping,
CMake will think the command is ended and the user is trying to do something
new.

That's the first and most important step.  The above functionality would
still be inconvenient to use (e.g. cmake -E will not be aware of any
compiled EXE target output paths).  So, enhancing e.g. add_custom_command as
I proposed might make it easier, as this command could be aware of compiled
EXE targets.  Finally, we'd want to add features to ExternalProject that use
the new CMake functionality to finally solve the original problem.

The alternative ideas in my e-mail are just a question of who generates
the above temporary script that sources/calls a user script.  E.g. in
option 3, we leave C++ portion of CMake alone, and ExternalProject would
then have to auto-generate the above temporary script instead.

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community

[cmake-developers] Setting up environment using ExternalProject_Add

[James Johnston]

, and probably option 1a.  In that
case, steps would be: (1) implement cmake -E run_commands, (2) extend
add_custom_command, (3) extend ExternalProject_Add.  What do you think?

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] Silencing warning about environment when using NMake / JOM

[James Johnston]

Hi,

While using the NMake and JOM generators in CMake, I have come across this
warning:

  To use the [NMake | JOM] generator, cmake must be run from a shell that
can use the
  compiler cl from the command line.  This environment does not contain
  INCLUDE, LIB, or LIBPATH, and these must be set for the cl compiler to
  work.

OK, so this is kind of a minor complaint, but this is an annoying warning if
you aren't actually wanting Visual C++ in your environment.  What if I was
wanting to use those generators without setting up a Visual C++ environment
with vcvarsall.bat?  Example use cases where this warning is annoying:

 * Using it to invoke a superbuild where: (1) the superbuild itself does not
use any languages (project command says NONE for languages), (2) the
superbuild sets up the VCVars environment uniquely for each
ExternalProject_Add.  For example, I might have ExternalProject_Add to build
both 32-bit and 64-bit versions of an ExternalProject.  And so I'd have
ExternalProject invoke VCVarsAll.bat uniquely for each environment.  In this
situation, running VCVars to silence the warning before invoking superbuild
is NOT good, because now I have two nested VCVars environments: (1) if the
VCVars invoked by the superbuild fails, it will be masked by the first call
to VCVars, meaning you could end up building for the wrong environment -
without error (VCVars doesn't return proper error codes), (2) and now you
have a PATH/LIB/INCLUDE that has combined paths for both 32-bit and 64-bit
building.

 * Using the NMake/JOM generators with non-Visual C++ compilers (which DOES
seem to work without issue otherwise).  In that case, of course I'm not
going to run VCVarsAll as a preliminary step...

I examined the CMake source code and found out that if either the INCLUDE or
LIB environment variables are missing, then the warning is emitted.  So I've
worked around it for now by setting both variables to a semicolon ; in my
environment.  But this hack doesn't pass the smell test for me, starting
with the fact that it involves messing with my environment in a weird way
and ending with relying on an undocumented detail of CMake.

Can these warnings be either eliminated entirely or suppressed via a more
documented method?  I could probably provide a patch but I'm not sure which
direction to take with it.

Best regards,

James Johnston


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] [PATCH] nmake/jom: Only warn about bad VS environment if compiler not found

[James Johnston]

Hi,

Attached is a patch that changes the warning of nmake/jom generators when
VCVarsAll.bat hasn't been run:

   nmake and jom generators no longer warn about missing INCLUDE/LIB
   environment variables unless the C/CXX compiler cannot be found.  This
   is useful if the user does not want to use these generators with the
   Visual C++ compiler, or they do not want to enable any language.

Sample output; this was run from a clean cmd.exe - i.e. without having run
VCVarsAll.bat:

C:\Users\JamesJ\AppData\Local\Temp\cmake-warn-build..\cmake-build\bin\cmake
-GNMake Makefiles JOM ../cmake-warn
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

cl

  is not a full path and was not found in the PATH.

  To use the JOM generator with Visual C++, cmake must be run from a shell
  that can use the compiler cl from the command line.  This environment is
  unable to invoke the cl compiler.  To fix this problem, run cmake from the
  Visual Studio Command Prompt (vcvarsall.bat).

  Tell CMake where to find the compiler by setting either the environment
  variable CC or the CMake cache entry CMAKE_C_COMPILER to the full path
to
  the compiler, or to the compiler name if it is in the PATH.


CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

cl

  is not a full path and was not found in the PATH.

  To use the JOM generator with Visual C++, cmake must be run from a shell
  that can use the compiler cl from the command line.  This environment is
  unable to invoke the cl compiler.  To fix this problem, run cmake from the
  Visual Studio Command Prompt (vcvarsall.bat).

  Tell CMake where to find the compiler by setting either the environment
  variable CXX or the CMake cache entry CMAKE_CXX_COMPILER to the full
path
  to the compiler, or to the compiler name if it is in the PATH.


-- Configuring incomplete, errors occurred!
See also
C:/Users/JamesJ/AppData/Local/Temp/cmake-warn-build/CMakeFiles/CMakeOutput.
log.
See also
C:/Users/JamesJ/AppData/Local/Temp/cmake-warn-build/CMakeFiles/CMakeError.l
og.

Best regards,

James Johnston


0001-nmake-jom-Only-warn-about-bad-VS-environment-if-comp.patch
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] [PATCH] Fix bug where CACHE ignored by get_filename_component

[James Johnston]

Hi,

Multiple patches attached that improve testing of the get_filename_component
command and also fix a bug where get_filename_component ignores CACHE when
PROGRAM_ARGS is also provided.

Best regards,

James Johnston


0001-get_filename_component-Tests-now-check-for-proper-CA.patch
Description: Binary data


0002-get_filename_component-Added-initial-tests-for-PROGR.patch
Description: Binary data


0003-get_filename_component-Fix-bug-where-CACHE-was-ignor.patch
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] [PATCH] string: State return value if string(FIND) doesn't find a match

[James Johnston]

Very minor documentation update clarifying what happens if string(FIND)
doesn't find a match.  I had to make a CMake test script and/or check CMake
source code to find the answer.

Best regards,

James Johnston



0001-string-State-return-value-if-string-FIND-doesn-t-fin.patch
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] malware?

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On 24/07/15 07:04, David Powell wrote:
  hi
 
  I downloaded cmake an hour ago from cmake.org http://cmake.org/ and
 found myself with an unwanted piece of software called advanced mac
 cleaner, an app that was hard to get rid of. I'm not certain it came from
your
 site but it happened at the same time and I can't think of any other
 explanation..  The download file from cmake.org http://cmake.org/
 (supposedly the latest stable dmg for mac) was much bigger (30MB) than the
 cmake file I subsequently downloaded from github.
 
 
 
 
 I don't know about that, but I just noticed that cmake.org allows HTTP
 (non-HTTPS) downloads.
 
 HTTP has no form of cryptographic authentication or verification, and it's
 incredibly easy for a MitM to attach malware to your downloads.
 
 IMO, the HTTP downloads should be removed ASAP.

Two other ideas that don't require HTTPS hosting of large binary files:

 * On Windows, cryptographically sign the setup program using Authenticode.
When the UAC prompts for elevation, Windows will show it signed by Kitware
instead of a yellow warning Unknown.  Probably the other operating systems
have a first-class way of doing something like this as well.  Downside:
certificates cost some modest amount of money to renew every year.

 * Post SHA-1 hashes of the EXEs/DMGs/tarballs on the CMake web site, and
post them over HTTPS.  But downside here is that many users won't bother to
check this (e.g. Windows has no well-known in-built utility for calculating
a file hash).

I agree the current situation of unsigned files available over HTTP only is
not really ideal.  Perhaps this would be a good opportunity for looking at
enhancements to CMake itself in the area of code signing (e.g. code signing
of individual target EXEs/DLLs, and code signing of the final setup EXE
package by CPack) that hides the various operating-system-specific ways of
doing this?  Then, CMake itself can be modified to be built with these new
features, if available.  A quick Google search of cmake.org for code signing
didn't yield much in the way of previous discussion or existing features...

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [MODERN] CMake 3.3 vs Qt 5.5 vs MSVC2015

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of Nils Gladitz
 
 On 07/23/2015 03:47 PM, Konstantin Podsvirov wrote:
  http://blogs.msdn.com/b/vcblog/archive/2015/03/03/introducing-the-univ
  ersal-crt.aspx
 
  CMake uses the module InstallRequiredSystemLibraries to install
 dependencies.
  Who is in charge here on this module? Should this module to install
  Universal CRT (api-ms-win-crt-runtime-xxx.dll library) for MSVC14 ?
 
6. App-local deployment of the Universal CRT is not supported.
 
 I think that means installation should not be handled by
 InstallRequiredSystemLibraries.
 
 You should probably just require users to run Windows Update.

That sounds horrible - asking a user to manually run Windows Update.  But
Windows Update packages don't have to be installed ONLY by way of visiting
Windows Update manually.

I have yet to mess with this stuff myself (but will be soon).  But in the
meantime, FTA:

Currently these MSU packages are installed as part of the VCRedist
installation. In a future build of Visual Studio 2015, these MSU packages
will also be distributed separately as part of the Universal CRT SDK and
made available for download on support.microsoft.com.

So you get this just by running VCRedist - or, if you don't use VCRedist,
you can get this by running the MSU packages directly (once said SDK becomes
available - if it isn't already available - the blog is 4 months old).  Both
can be accomplished as part of a setup program.  Unfortunately, I suspect
the MSU packages can't be invoked from / nested inside a Windows Installer
MSI like how the VC++ merge modules were done (and how we currently do); I
plan to hack apart VCRedist to try to see what they are doing...

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [MODERN] CMake 3.3 vs Qt 5.5 vs MSVC2015

[James Johnston]

 -Original Message-
 From: Nils Gladitz [mailto:nilsglad...@gmail.com]
 
 On 23.07.2015 17:24, James Johnston wrote:
  That sounds horrible - asking a user to manually run Windows Update.
  But Windows Update packages don't have to be installed ONLY by way of
  visiting Windows Update manually.
 
 But this will only be an issue for users which don't run windows = 10 or
 which don't run regular updates (I'd have assumed this comes in through
 automated updates as well unless disabled?).

Depends on if it is an important update or an optional update...

Or it may not show at all.  E.g. the legacy WinHelp MSU package is not
pushed via Windows Update at all, but if you want WinHelp, it's installed
via a manually-downloaded MSU package that you have to get from
Microsoft.com downloads.

I'm curious to find out which method they choose...

 I rather think it is nice that this is considered an operating system
component
 rather than a visual studio version specific runtime component (e.g.
reduced
 application installer sizes and centralized updates) but this might work
best if
 you also treat it as such (unless you specifically have to target e.g.
 outdated/unnetworked machines).

To me, this is the worst of both worlds.  Not only do we STILL have to
distribute the version-specific components (which are not part of the
universal CRT) as we always have since VS2005, we now have this MSU package
IN ADDITION that has to be installed for every configuration except static
runtime linking.

You still have the burden of redistributing the rest of the unstable
VS2015-specific runtime.  So it's not like you can wait 10 years for
adoption and stop worrying about any VS2015 redistributing runtime files at
all.

Bonus points if they find a way in Visual Studio 2020 to modify the
universal CRT and somehow cause breaking changes with VS2015, even though
they promised not to...  This was supposedly why the VS2005 CRT  newer are
side-by-side installed into winsxs - to fix DLL hell from the 1990s...

James

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] [PATCH] ExternalProject: Added new USES_TERMINAL options

[James Johnston]

Hi,

I worked on a patch to allow you to pass USES_TERMINAL through
ExternalProject to the underlying add_custom_command().  Here is the commit
message:

ExternalProject:  Added new USES_TERMINAL options

Added new USES_TERMINAL option to the ExternalProject_Add_Step
function.  This option passes USES_TERMINAL to the underlying
add_custom_command call so that the Ninja console pool is used.
Also, corresponding new USES_TERMINAL_step options were added
to the ExternalProject_Add function.

Justification: if using Ninja with a CMake superbuild, it's often
desirable to limit the superbuild to ONE sub-Ninja process at a
time to avoid oversubscribing the CPU.  Using the console pool also
makes it easy to monitor the progress of the sub-Ninja process.

Independent USES_TERMINAL_step arguments are passed to
ExternalProject_Add instead of one USES_TERMINAL argument that
controls everything.  Users may wish to run some steps in parallel
but not others (e.g. parallelize configure but not build).

Best regards,

James Johnston


0001-ExternalProject-Added-new-USES_TERMINAL-options.patch
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] C# support?

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of Brad King
 Sent: Tuesday, June 30, 2015 13:49
 To: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] C# support?
 
 On 06/30/2015 03:21 AM, Stuermer, Michael SP/HZA-ZSEP wrote:
  it would be great if some people could step forward once everything is
  running from my side to help get makefile and linux support (and test
  other Visual Studio versions).
 
 Once you have it working in VS 2013 the other VS = 2010 versions should
be
 easy.  I'm not sure about other generators yet.  If you have good coverage
in
 the test suite updates then that will make the task of adding support to
other
 generators easier.

What about Visual Studio 2005 / 2008?  In those scenarios:
 * The Visual C++ projects are NOT MSBuild projects (as the CMake devs I'm
sure well know)
 * The Visual C# project *is* an MSBuild project.

My guess is if targeting VS2005/2008, you'd need to somehow use code as
follows:
 * The Visual C++ 2005/2008 projects generated as normal using
VisualStudio8/9TargetGenerator classes.
 * The Visual C# 2005/2008 projects are MSBuild so you'd have to use the new
code in VisualStudio10TargetGenerator when your CMake generator was set to
Visual Studio 8/9?  It sounds goofy...
 * And all the above projects put into one final VS2005/2008 solution file.

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] C# support?

[James Johnston]

 -Original Message-
 From: cmake-developers [mailto:cmake-developers-boun...@cmake.org]
 On Behalf Of David Cole via cmake-developers
 Sent: Monday, June 29, 2015 17:31
 To: Brad King
 Cc: cmake-developers@cmake.org
 Subject: Re: [cmake-developers] C# support?
 
 The C# compiler, csc.exe, takes all its arguments at once in one call to
build a
 library or executable. Listing all the sources, and its references (other
libraries
 it depends on) all at once. You can do it as command line arguments, or as
 contents of a response file, or a combination or arguments plus response
file.
 
 Conceptually, it's just like Java.

Exactly right; it's like a combined compiler+linker.  It's also worth noting
that the C# and VB .NET languages doesn't have the concept of #include
preprocessor directives.  Header files are a non-issue.  (Once you got that
working, I'd imagine Java language support would be easy, too...)

 They do have separate project files for it with VS, though. The generators
will
 need code to generate *.csproj files, rather than custom commands in a
 vcxproj file, to make it seem like it's really well-integrated with VS.
Not sure if
 *.csproj files have evolved much over the last few releases of VS -- I'd
expect
 the major challenge with this to be making sure CMake generates proper
 *.csproj files for however many versions of VS it would take to make it
 acceptable.

Csproj files are MSBuild compatible, like how Visual C++ 2010 vcxproj files
are.  While Visual C++ went to MSBuild recently (VS2010), Visual C# has been
MSBuild compatible since MSBuild's inception - Visual Studio 2005 if I
remember right.  Visual Studio 2002/2003 might be different since they
pre-date MSBuild but since the VC++ 2002 generator was recently deprecated
I'd argue these versions can be ignored. (if someone uses that version
still, they could at least use csc.exe directly via a Makefile generator)

So, since VS2005, I think you'd find small changes (new flags introduced,
etc.) but no major file format overhaul like from VC++ 2008 to VC++ 2010.

I'd also like to take this opportunity to point out that whoever does this
might want to consider supporting VB .NET also.  Everything in this
discussion is EXACTLY the same, except that the compiler name is different,
and project/source code file extensions are different.  It would be
relatively trivial to support VB .NET once you have C# working, if you wrote
the generator right.  (vbc.exe instead of csc.exe, vbproj instead of csproj,
.vb file instead of .cs file.  Compiler switches are probably almost
identical...  It's also MSBuild-based and also does a combined
compile+link.)

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] C# support?

[James Johnston]

 -Original Message-
 From: CMake [mailto:cmake-boun...@cmake.org] On Behalf Of Stuermer,
 Michael SP/HZA-ZSEP
 Sent: Friday, June 26, 2015 14:47
 To: cmake-developers@cmake.org; cm...@cmake.org
 Subject: [CMake] C# support?
 
 Hi and sorry for cross-posting this on both lists,
 
 I checked the mailing list history about the C#/.NET support topic and
realized
 that the interest in C# support seems to have declined a bit.
 
 I am right now in the need of good C# support and adding external project
 files is not that much of an option to me. So I started hacking away
 everything that is needed for .csproj generation and support of mixed
 managed/unmanaged targets. Not yet done, but there is a light at the end
of
 the tunnel.
 
 Now the question: is there any real interest at all in this feature? Does
it have
 a realistic chance to be accepted for upstream or will I have to maintain
my
 own fork of CMake?
 

Well, I have only just recently started using CMake.  So I am not speaking
from a great deal of experience.  But I am currently converting over a bunch
of stuff into a set of CMake projects.  (must be multiple CMake projects
built by a superbuild via ExternalProject because we use three different
C/C++ compilers and target three different CPU architectures.) 

Part of that stuff involves a mixture of Visual Studio projects involving
(1) 100% unmanaged C++ code: this is the easy stuff to move to CMake, (2)
C++/CLI mixed mode projects, (3) C# projects that consume the mixed mode
C++/CLI projects.  I have yet to start working on that part of the CMake
migration, but I'm not looking forward to figuring out how to make the C#
projects consume the build output from the C++/CLI projects and still have
the Visual C# projects be easy to add/remove files from.  All these
particular projects are targeting Visual Studio 2008 32-bit, so it *ought*
be possible to use them all from one single CMake build tree  one single
Visual Studio solution...

I have seen include_external_msproject CMake command:
http://www.cmake.org/cmake/help/v3.3/command/include_external_msproject.html
But it's not clear to me how I'd make the C# project depend on other CMake
targets (i.e. the mixed mode C++ projects).  The best I've thought of would
be to use configure_file to modify the project to substitute the compiled
C++ DLLs in the references section of the project.  But then it will be a
pain to modify the original C# projects in SOURCE_DIR.  And also, the C#
project normally has relative paths, so if it gets configure_file'd into
BINARY_DIR all the source code paths will break.

I guess all the above is to say, I'm interested in that feature.  But I may
be too much of a newbie at CMake to say whether that is a good CMake way
of doing things.  I still need to experiment with the above command, and
read up on old mailing list messages on the subject.

But it seems to me the logical way for supporting C# would be: (1) it
becomes a first-class language that can be mentioned in the project()
command, (2) therefore, all the generators can use it, so you aren't
restricted to Visual Studio generator (I like the Ninja generator because
it's faster than Visual Studio), (3) Visual Studio generator would emit a
solution containing generated VC++/VC# projects.  Anything else feels like a
bit of a hack to me.  But #1 thru #3 sounds like a lot of work though...
Also, C# doesn't compile to object files - does CMake currently support the
concept of a language that doesn't require separate compiling/linking steps?

Best regards,

James Johnston

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [PATCH] Fixed Borland linker issue when used with Ninja generator

[James Johnston]

See updated attached patch.  I've retested it to ensure the VTK 5.4.2 build
I've been working with still builds correctly - and it still does.

Thank you for considering it.

-James

 -Original Message-
 From: Brad King [mailto:brad.k...@kitware.com]

  +set(CMAKE_JOB_POOL_LINK BCC32LinkPool)
 
 Hmm.  What if the user sets this too?  Perhaps we should allow
 users/projects to take control of this setting by making the whole block
 conditional:
 
   if(NOT DEFINED CMAKE_JOB_POOL_LINK)

Fair enough.  Change made.  I can't think of a real-world reason why someone
would want to do this (given the buggy behavior that will result), but I
suppose it's better not to step in the way.  (The only thing I could think
of would be if you wanted both compiler and linker to use the SAME job pool,
which would then have to be restricted to 1.  At that point, there's no
reason to using Ninja in the first place...)

 
  +get_property(_EMB_TMP_POOLS GLOBAL PROPERTY JOB_POOLS)
 list(APPEND
  +_EMB_TMP_POOLS BCC32LinkPool=1) set_property(GLOBAL PROPERTY
  +JOB_POOLS ${_EMB_TMP_POOLS})
 
 This can be just
 
  set_property(GLOBAL APPEND PROPERTY JOB_POOLS BCC32LinkPool=1)

Oops, I overlooked that!  Much cleaner now, thanks :)



0001-Work-around-Borland-linker-issue-when-used-with-Ninj.patch
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] [PATCH] Fixed Borland linker issue when used with Ninja generator

[James Johnston]

Hi,

The attached patch is in regards to issue #15620 in mantis bug tracker
(Ninja generator triggers race condition in Borland bcc32 linker causing
intermittent build failure).  Thanks to Brad King's suggestion, I think
using job pools is a lot easier than trying my proposed workaround (which I
realized would probably involve more disruptive code changes).  (I'm new to
both CMake/Ninja, so wasn't aware of this option).

I've tested with building VTK 5.4.2.  Before the patch, 5 out of 5 builds
failed (sooner or later, in a big project like VTK, the race condition would
trash the build).  After the patch, 5 out of 5 builds work without error.

Because of this issue with the linker itself, I can't think of any reason
why anybody would want to change the default settings I put in
Modules/Platform/Windows-Embarcadero.cmake with this patch.  You simply
*must* restrict the linker to one instance, without exception, or you'll
have problems due to the bcc32 linker design flaws I've outlined in the bug
tracker and patch comments.  I was only able to think of three solutions to
this problem:

1.  Modify each upstream CMake project (e.g. VTK) itself to set the Ninja
job pool in the Borland case (not a good option).
2.  Provide a toolchain file that sets up the job pool.  Awkward, since you
have to do this EVERY TIME you use the Ninja generator /w bcc32 or it WILL
NOT WORK.
3.  Patch Windows-Embarcadero.cmake, as I propose here.

(Since JOB_POOLS is a global property and not a variable, I couldn't find a
way to simply pass this as a parameter to cmake.exe - not that that would be
an ideal solution anyway.)

Best regards,

James Johnston



0001-Work-around-Borland-linker-issue-when-used-with-Ninj.patch
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers