Roger Leigh edited comment on XERCESC-2077 at 7/14/16 11:55 PM:

Unrelated to the patch, but the extensive CI testing showed a few problems 
which may have been missed.  Some of the compiler warnings may warrant further 
investigation such as macosxunicodeconverter calling sizeof on an array 
function argument, and there may also be an issue with the thread tests on 
FreeBSD.  I'll file separate tickets, with patches where possible, as I 
investigate them in more detail.

was (Author: rleigh):
Unrelated to the patch, but the extensive CI testing showed a few problems 
which may have been missed.  Some of the compiler warnings may warrant further 
investigation such as macosxunicodeconverter calling sizeof on an array 
function argument, and there may also be an issue with the thread tests on 
FreeBSD.  I'll files separate tickets, with patches where possible, as I 
investigate them in more detail.

> Add CMake build system
> ----------------------
>                 Key: XERCESC-2077
>                 URL: https://issues.apache.org/jira/browse/XERCESC-2077
>             Project: Xerces-C++
>          Issue Type: New Feature
>          Components: Build
>    Affects Versions: 3.1.4
>         Environment: All
>            Reporter: Roger Leigh
>              Labels: build, cmake, patch
>         Attachments: 0001-cmake-Add-CMake-build-system.patch
> h4. Introduction
> The attached patch implements a CMake build for Xerces-C++.
> I have spent significant effort performing a "comprehensive" conversion of 
> the existing GNU autotools and MSVC project file logic to a unified CMake 
> build which supports all platforms with a single set of build files, as well 
> as testing it exhaustively (see below). The existing GNU autotools build and 
> MSVC project builds will continue to function and are unaffected by this 
> addition.
> h5. References
> - http://mail-archives.apache.org/mod_mbox/xerces-c-dev/201302.mbox/browser
> - http://mail-archives.apache.org/mod_mbox/xerces-c-dev/201506.mbox/browser
> - https://github.com/rleigh-codelibre/xerces-c/tree/cmake-3.1
> h4. Background
> CMake is a meta-build system which generates the build files for a specified 
> build system, such as make, Visual Studio msbuild, nmake, ninja or a number 
> of other build tools and IDEs.  This allows Xerces-C++ to be built on any 
> supported platform with the native tools for that platform.
> The reason why I originally needed this was due to the large maintenance 
> burden of patching the provided Visual Studio project files, both for fixing 
> bugs in those files and in being able to support versions of Visual Studio 
> which aren't yet supported by the provided project files or for unsupported 
> configurations e.g. Clang/C2, other platforms etc.  The lack of an install 
> target also meant that to integrate this with a larger build required 
> manually copying bits out of the build tree.  The cost of debugging and 
> patching the existing project files for use in our CI builds was getting too 
> great--maintaining and using this CMake build out of tree will be cheaper and 
> more robust.  However, given that other people have also requested such 
> support in the past, I thought it might benefit others to have this merged 
> upstream so that it would be available to the benefit of all.
> I have done a direct conversion of every autoconf option and feature test.  
> Where there wasn't a direct CMake equivalent, I've written each feature test 
> to exactly match the autoconf behaviour.  The automake Makefile.am logic is 
> directly represented in the corresponding CMakeLists.txt files.  Broadly:
> ||Autotools||CMake||
> |{{configure.ac}}, {{Makefile.am}}|{{CMakeLists.txt}}|
> |{{*/Makefile.am}}|{{*/CMakeLists.txt}}|
> |{{m4/*}}|{{cmake/*}}|
> |{{src/xercesc/util/Xerces_autoconf_config.hpp.in}}|{{src/xercesc/util/Xerces_autoconf_config.hpp.cmake.in}}|
> |_autoheader_|config.h.cmake.in|
> |{{tools/createdocs.sh}}|{{CMakeLists.txt}} (custom target)|
> |{{scripts/sanityTest.pl}}|{{cmake/XercesTest.cmake}} (direct support)|
> |{{scripts/sanityTest_ExpectedResult.log}}|{{test/expected/\*}}, 
> {{samples/expected/\*}} (individual log files)|
> And there's a section added to the documentation giving an overview of how to 
> use it, in the same style as the autotools section.
> h5. Enhancements over the existing build systems
> - Universal build for any platform and build system supported by CMake
> - Full support for feature and library detection on Windows, including
>   discovery of ICU libraries; it's no longer static, using (long broken)
>   ICU configurations in the project files
> - An install target now exists on Windows, so the various pieces don't
>   need manually copying out of the build tree
> - Parallel build speed improvements when using ninja to replace make
>   or msbuild; the speedup with the latter is significant
> - Export of CMake configuration in addition to pkg-config, to make
>   Xerces-C++ integrate with downstream projects using Xerces-C++ and
>   cmake; this includes all dependency information of the libraries
>   Xerces was linked with, i.e. transitive dependencies.
> - Installs the HTML documentation
> - Targets are provided for regenerating the documentation (docs and
>   apidocs)
> - Documentation can be edited and rebuilt from within Visual Studio
> - Unit tests can be run on all supported platforms
> - Unit tests can be run in parallel
> - Unit tests verify individual test output on all platforms
> - Unit tests can be run from within Visual Studio
> - All the Visual Studio projects are grouped into categories
>   (Documentation, Library, Samples, Tests), making it neater and
>   easier to navigate than with the existing solution files
> h5. Known differences:
> - It's not using libtool; all the library generation, version
>   numbering and naming is done by cmake using its default conventions
>   for each platform.  This results in slightly different library
>   names, depending upon the platform and library type.  The library
>   link name is xerces-c on all platforms.  This could potentially be
>   tweaked to match the pre-existing naming where strictly required,
>   though being able to use consistent naming in cross-platform
>   projects is also quite advantageous.
> h4. Maintenance
> The logic in all files directly matches the corresponding autotools files to 
> the maximum extent possible.  For most updates to the autotools logic, the 
> corresponding cmake change should be trivial and obvious, for example adding 
> or removing source files from src/Makefile.am or altering the supported 
> options.  The generated headers are almost identical, and so the build should 
> be exactly the same as with the autotools.  Every feature test and define has 
> been checked.
> The CMake build parses the versioning information directly from configure.ac 
> and version.incl, so no updates are required for releasing.
> If you want or need an ongoing maintenance commitment from myself, I can 
> certainly provide it.  It is quite possible we can also provide some capacity 
> for continuous integration, for example see the [TIFF 
> jobs|https://ci.openmicroscopy.org/view/Third-Party/] we currently host, 
> amongst others, which test with a number of platforms and build systems on an 
> ongoing basis.
> h4. Comparison of build system sizes
> ||system||lines||chars||bytes
> |autotools|4023|11711|142926|
> |cmake|4608|13807|159496|
> |VC12|24400|37475|1465400|
> |VC14|25551|41265|1553303|
> Obtained using:
> {noformat}
> wc  CMakeLists.txt config.h.cmake.in 
> src/xercesc/util/Xerces_autoconf_config.hpp.cmake.in  */CMakeLists.txt cmake/*
> wc configure.ac Makefile.am src/xercesc/util/Xerces_autoconf_config.hpp.in 
> */Makefile.am m4/* scripts/sanityTest.pl tools/createdocs.sh reconf 
> config/pretty-make
> find projects/Win32/VC12 -type f | xargs wc
> find projects/Win32/VC14 -type f | xargs wc
> {noformat}
> It's not too dissimilar to the autotools; given that it's a direct copy of 
> the same logic and also implements additional functionality, this isn't too 
> surprising.  But it's almost 10 times smaller than the Visual Studio builds, 
> and this is multiplied by every supported Visual Studio version, which 
> doesn't scale well in either space or maintenance effort required to keep 
> every version up to date and consistent between versions.  The CMake build 
> can generate project files for every Visual Studio version, as well as nmake, 
> ninja and other supported build tools on Windows, making it vastly more 
> flexible in addition to being much more maintainable.  The static VC project 
> files don't have the degree of configurability which the CMake generation 
> brings.
> While this patch does not remove the VC project files, that would be one 
> potential follow up for the trunk should that be desired.
> h4. Test builds
> The build has been tested extensively on several platforms, exercising all 
> major configuration options:
> - 
> [FreeBSD|https://ci.openmicroscopy.org/view/Third-Party/job/XERCESC-CMAKE-BSD/]
> - 
> [Linux|https://ci.openmicroscopy.org/view/Third-Party/job/XERCESC-CMAKE-LINUX/]
> - [MacOS 
> X|https://ci.openmicroscopy.org/view/Third-Party/job/XERCESC-CMAKE-MACOSX/]
> - [Windows - Visual 
> Studio|https://ci.openmicroscopy.org/view/Third-Party/job/XERCESC-CMAKE-WINDOWS/]
> - [Windows - 
> Cygwin|https://ci.openmicroscopy.org/view/Third-Party/job/XERCESC-CMAKE-CYGWIN/]
> - [Windows - 
> MinGW64|https://ci.openmicroscopy.org/view/Third-Party/job/XERCESC-CMAKE-MINGW64/]
> Build logs are available for every configuration variant for each platform.  
> It's passing in all cases for every variation of every platform.
> All platforms test:
> - Release and Debug builds
> - Shared and Static libraries
> Additional options:
> h6. BSD
> - make and ninja generators
> {noformat}
>   1: -Dnetwork:BOOL=ON -Dnetwork-accessor=curl -Dmessage-loader=icu 
> -Dtranscoder=iconv
>   2: -Dnetwork:BOOL=ON -Dnetwork-accessor=socket -Dmessage-loader=inmemory 
> -Dtranscoder=icu
>   3: -Dnetwork:BOOL=ON -Dmessage-loader=iconv
>   4: -Dnetwork:BOOL=OFF
>   5: none (autodetected defaults used)
> {noformat}
> h6. Linux:
> -make and ninja generators
> {noformat}
>   1a: -Dnetwork:BOOL=ON -Dnetwork-accessor=curl -Dmessage-loader=icu 
> -Dtranscoder=iconv
>   1b: -Dnetwork:BOOL=ON -Dtranscoder=iconv
>   2: -Dnetwork:BOOL=ON -Dnetwork-accessor=socket -Dmessage-loader=inmemory 
> -Dtranscoder=icu
>   3a: -Dnetwork:BOOL=ON -Dnetwork-accessor=socket -Dmessage-loader=inmemory
>   3b: -Dnetwork:BOOL=ON -Dmessage-loader=iconv
>   4: -Dnetwork:BOOL=OFF
>   5: none (autodetected defaults used)
>   (a - Ubuntu 14.04 with CURL and ICU; b - CentOS 7 without CURL and ICU)
> {noformat}
> h6. MacOS X
> - make and ninja generators
> {noformat}
>   1: -Dnetwork:BOOL=ON -Dnetwork-accessor=curl -Dmessage-loader=icu 
> -Dtranscoder=macosunicodeconverter
>   2: -Dnetwork:BOOL=ON -Dnetwork-accessor=socket -Dmessage-loader=inmemory 
> -Dtranscoder=icu
>   3: -Dnetwork:BOOL=ON -Dnetwork-accessor=cfurl -Dmessage-loader=iconv 
> -Dtranscoder=iconv
>   4: -Dnetwork:BOOL=OFF
>   5: none (autodetected defaults used)
> {noformat}
> h6. Windows
> - visual studio and ninja generators
> {noformat}
>   VS2013 and VS2015
>   1: -Dnetwork:BOOL=ON -Dmessage-loader=icu -Dtranscoder=iconv
>   2: -Dnetwork:BOOL=ON -Dnetwork-accessor=winsock -Dmessage-loader=inmemory 
> -Dtranscoder=icu
>   3: -Dnetwork:BOOL=ON -Dtranscoder=windows
>   4: -Dnetwork:BOOL=OFF
>   5: none (autodetected defaults used)
> {noformat}
> h6. Cygwin
> - make generator
> {noformat}
>   1: -Dnetwork:BOOL=ON -Dnetwork-accessor=curl -Dmessage-loader=icu 
> -Dtranscoder=iconv
>   2: -Dnetwork:BOOL=ON -Dnetwork-accessor=socket -Dmessage-loader=inmemory 
> -Dtranscoder=icu
>   3: -Dnetwork:BOOL=OFF
>   4: none (autodetected defaults used)
> {noformat}
> h6. MinGW64
> - ninja generator
> {noformat}
>   1: -Dnetwork:BOOL=ON -Dtranscoder=iconv
>   2: -Dnetwork:BOOL=ON -Dtranscoder=windows
>   4: -Dnetwork:BOOL=OFF
>   5: none (autodetected defaults used)
> {noformat}
> h4. Summary
> I hope that the extensive test matrix demonstrates that the build is fully 
> functional for all of the major platforms, for all of the configuration 
> options autoconf provided.  It is expected to be equally functional for minor 
> platforms as well.
> While it certainly improves upon the autotools build in some areas (unit 
> tests, documentation), the most significant benefit is for building on 
> Windows, where it brings up the build to feature parity with the Unix 
> platforms including all the same feature testing and library detection logic, 
> as well as bringing missing functionality such as a configurable installation 
> prefix, support for alternative build tools, and support for toolchains which 
> the project files do not support.  This is greatly desirable, and was the 
> primary driver for this work, as were the earlier requests for such support.
> I can appreciate that this is a rather large change to propose, but as an end 
> user of the library, it's one which I have greatly needed for several years, 
> but only had the time recently to dedicate a serious amount of my time to do 
> this with the attention to detail required to make this a conversion of the 
> highest quality.
> I do hope that anyone testing this finds it useful, usable and makes building 
> and installing Xerces-C++ a pleasure on any platform.
> Kind regards,
> Roger Leigh

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org

Reply via email to