Re: [xz-devel] cmake support?
On 2020-02-28 Mario Emmenlauer wrote: > It seems I did find an impediment, after all. I've back-ported the > CMakeLists.txt to xz-5.2.4 as described before. The only change > required was to remove two source files, > src/liblzma/common/file_info.c and src/liblzma/common/index_decoder.h. This should be fine for Windows. There are *minor* details why it's not perfect for other platforms but those don't matter here. > But on Windows with VS2017 latest and VS2019 latest, I have problems > with the processing of liblzma_w32res.rc. Below is the error message > from nmake, but it fails in a similar (not identical) way with ninja > build system. Great to see this tested. It was one thing that I felt unsure if it will work. Weirdly the error message doesn't say anything else than the return code which isn't very useful. > [ 98%] Building RC object > CMakeFiles/liblzma.dir/src/liblzma/liblzma_w32res.rc.res > C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe -DDLL_EXPORT [...] > -IC:\data\Debug\xz-5.2.4\src\common -DWIN32 -D_DEBUG /fo At least it seems to try to build it with the resource compiler and the include path had src/common in it so it will find common_w32res.rc too. Does rc.exe really print nothing? Can you run the command manually? Can you try if the VS project files under windows/vs* work for building a DLL? They should build the resource file too and I have an impression that the project files have been tested, so perhaps there are clues what is done differently (or if the resource file just isn't compatible). The project files have seen updates since 5.2.4 too, so please test with xz.git or using this snapshot: https://tukaani.org/temp/xz-5.3.1alpha-104-g265d.tar.xz > -DHAVE___BUILTIN_ASSUME_ALIGNED This sounds like a bug in the CMake files unless MSVC really supports __builtin_assume_aligned that GCC and Clang support. xz 5.2.4 doesn't recognize that define so it's harmless there. So this must be fixed too if it is broken. The above snapshot is good for testing this with CMake. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] cmake support?
On 27.02.20 20:52, Lasse Collin wrote: On 2020-02-27 Mario Emmenlauer wrote: Awesome! I've just tested the CMakeLists with the latest official release xz-5.2.4 and it works flawless so far. Anyway, more feedback from CMake users is welcome. The liblzma part should work on multiple operating systems; it's not Windows-only anymore. See the comment in the beginning of CMakeLists.txt in xz.git. It seems I did find an impediment, after all. I've back-ported the CMakeLists.txt to xz-5.2.4 as described before. The only change required was to remove two source files, src/liblzma/common/file_info.c and src/liblzma/common/index_decoder.h. The build works for me on CentOS 7, Ubuntu 18.04, and macOS 10.14 with XCode 11.3. But on Windows with VS2017 latest and VS2019 latest, I have problems with the processing of liblzma_w32res.rc. Below is the error message from nmake, but it fails in a similar (not identical) way with ninja build system. I tried debugging this, but there was nothing obvious that I could see. Since there are no major changes between xz-5.2.4 and current HEAD with respect to liblzma_w32res.rc and common.rc, I'm slightly confused what may go wrong. Does this ring a bell with anyone? [...] [ 98%] Building RC object CMakeFiles/liblzma.dir/src/liblzma/liblzma_w32res.rc.res C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe -DDLL_EXPORT -DHAVE_CHECK_CRC32 -DHAVE_CHECK_CRC64 -DHAVE_CHECK_SHA256 -DHAVE_DECODERS -DHAVE_DECODER_ARM -DHAVE_DECODER_ARMTHUMB -DHAVE_DECODER_DELTA -DHAVE_DECODER_IA64 -DHAVE_DECODER_LZMA1 -DHAVE_DECODER_LZMA2 -DHAVE_DECODER_POWERPC -DHAVE_DECODER_SPARC -DHAVE_DECODER_X86 -DHAVE_ENCODERS -DHAVE_ENCODER_ARM -DHAVE_ENCODER_ARMTHUMB -DHAVE_ENCODER_DELTA -DHAVE_ENCODER_IA64 -DHAVE_ENCODER_LZMA1 -DHAVE_ENCODER_LZMA2 -DHAVE_ENCODER_POWERPC -DHAVE_ENCODER_SPARC -DHAVE_ENCODER_X86 -DHAVE_IMMINTRIN_H -DHAVE_INTTYPES_H -DHAVE_MF_BT2 -DHAVE_MF_BT3 -DHAVE_MF_BT4 -DHAVE_MF_HC3 -DHAVE_MF_HC4 -DHAVE_STDBOOL_H -DHAVE_STDINT_H -DHAVE_VISIBILITY=1 -DHAVE__BOOL -DHAVE__MM_MOVEMASK_EPI8 -DHAVE___BUILTIN_ASSUME_ALIGNED -DMYTHREAD_VISTA -DPACKAGE_BUGREPORT=\"lasse.col...@tukaani.org\" -DPACKAGE_NAME="\"XZ Utils\"" -DPACKAGE_URL=\"https://tukaani.org/xz/\"; -DTUKLIB_FAST_UNALIGNED_ACCESS -DTUKLIB_SYMBOL_PREFIX=lzma_ -Dliblzma_EXPORTS -IC:\data\Debug\xz-5.2.4\src\liblzma\api -IC:\data\Debug\xz-5.2.4\src\liblzma\common -IC:\data\Debug\xz-5.2.4\src\liblzma\check -IC:\data\Debug\xz-5.2.4\src\liblzma\lz -IC:\data\Debug\xz-5.2.4\src\liblzma\rangecoder -IC:\data\Debug\xz-5.2.4\src\liblzma\lzma -IC:\data\Debug\xz-5.2.4\src\liblzma\delta -IC:\data\Debug\xz-5.2.4\src\liblzma\simple -IC:\data\Debug\xz-5.2.4\src\common -DWIN32 -D_DEBUG /fo CMakeFiles\liblzma.dir\src\liblzma\liblzma_w32res.rc.res C:\data\Debug\xz-5.2.4\src\liblzma\liblzma_w32res.rc NMAKE : fatal error U1077: 'C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe' : return code '0x1' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\bin\HostX64\x64\nmake.exe"' : return code '0x2' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\bin\HostX64\x64\nmake.exe"' : return code '0x2' Stop. All the best, Mario Emmenlauer
Re: [xz-devel] cmake support?
On 2020-02-27 Mario Emmenlauer wrote:
> Awesome! I've just tested the CMakeLists with the latest official
> release xz-5.2.4 and it works flawless so far.
Thanks!
I got some off-list feedback about the CMake files in xz.git so I know
that at least something about them works (static builds with MSVC and
MinGW). Some details are unclear to me but I guess someone will complain
if there are problems, although problems that only affect performance
are unlikely to be detected without actually looking for such details
("appears to work" doesn't mean that it works *well*).
Anyway, more feedback from CMake users is welcome. The liblzma part
should work on multiple operating systems; it's not Windows-only
anymore. See the comment in the beginning of CMakeLists.txt in xz.git.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] cmake support?
Dear Lasse, On 24.02.20 23:12, Lasse Collin wrote: Hello! I committed some CMake support. It should support a few platforms (not just Windows) for building liblzma but it's not tested. Silly bugs are very much possible. The CMakeLists.txt in the previous email had a bug in the defines: they lacked quotation marks for the strings. The questions in that email are still valid. In addition to testing with MSVC, it can now be tested on other platforms too. At least I would like to know if it works with MSVC to build static and shared liblzma, and does it name the shared liblzma file "liblzma.dll" as I hope it does. Awesome! I've just tested the CMakeLists with the latest official release xz-5.2.4 and it works flawless so far. I had to remove two source files (src/liblzma/common/file_info.c and src/liblzma/common/index_decoder.h) and everything else seems to have worked out of the box. Whats more, I've seen that the previous cmake build was missing a large number of source files. I guess they are not mandatory but it feels good to have them included now :-) Thanks a lot again! All the best, Mario Emmenlauer
Re: [xz-devel] cmake support?
Hello! I committed some CMake support. It should support a few platforms (not just Windows) for building liblzma but it's not tested. Silly bugs are very much possible. The CMakeLists.txt in the previous email had a bug in the defines: they lacked quotation marks for the strings. The questions in that email are still valid. In addition to testing with MSVC, it can now be tested on other platforms too. At least I would like to know if it works with MSVC to build static and shared liblzma, and does it name the shared liblzma file "liblzma.dll" as I hope it does. Thanks! -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] cmake support?
On 2020-02-17 Mario Emmenlauer wrote:
> I think the conan community also enjoyes cmake because it integrates
> very well. So, since this CMakeLists.txt is quite well developed its
> not unlikely that more people looking mostly for liblzma will try to
> switch to cmake.
OK. I hadn't thought much about other use cases than package managers
included in the operating systems. With them one wants to build the
whole package anyway. Windows was a special case on top of this and
largely done as a portability exercise as LZMA SDK has always had great
Windows support. So these "only liblzma" cases are still a bit new
thought to me.
> Are you still considering this mostly for MSVC? If so, it may be
> helpful to add an explicit check, to avoid people trying to track
> down problems of missing 'config.h'.
Yes.
> But if more platforms could qualify it would certainly be
> appreciated! Maybe some of the major platforms like Linux and macOS?
Perhaps some day but let's get MSVC support done first (and a little
GNU/Linux too to let me test the basics).
I made a few changes to CMakeLists.txt:
- Added the header files to the add_library call.
- In target_compile_definitions, NDEBUG was replaced with
$<$:NDEBUG> to define NDEBUG only when no build type
has been specified. When e.g. "Release" is used, it already takes
care of defining NDEBUG.
- Use CMAKE_CURRENT_SOURCE_DIR instead of CMAKE_CURRENT_LIST_DIR when
appropriate. Only liblzmaConfig.cmake needs CMAKE_CURRENT_LIST_DIR.
- Added a commented-out alternative for -Wl,--version-script usage
that is compatible with CMake 3.1.
I didn't add a check for "NOT WIN32" yet.
> Anything I can do to help?
Yes, I have a few questions about the attached current CMakeLists.txt
when used with xz.git.
Does it work with MSVC targeting x86, x86-64, and ARM64?
Does the liblzma_w32res.rc get included in the build and does it build
correctly when creating liblzma.dll? You should see the information in
the file properties/details in Windows Explorer.
Optimizations in src/liblzma/common/memcmplen.h:
- If SSE2 support is enabled in MSVC settings and targeting 32-bit
x86, is the SSE2 version used in memcmplen.h? That is, are the
hairy #ifdefs correct for MSVC and does the HAVE__MM_MOVEMASK_EPI8
detection in CMakeLists.txt work with MSVC?
- On x86-64 the 64-bit non-SSE2 version should be used instead.
- The 64-bit version could be the best version for ARM64 too
but no one has tested it (it affects compression speed, not
decompression). Or perhaps some ARM64-specific code would be
even better. If no one can test this now, it's OK, but liblzma
should at least be buildable for ARM64. That is, CMake must *not*
define HAVE__MM_MOVEMASK_EPI8 for ARM64 builds.
Is the enable-uwp-builds.patch needed because using GetModuleHandle()
doesn't work for UWP applications? Like I said, I'll fix it but first I
want to know what is being fixed. :-)
CMakeLists.txt generates a few .cmake files and installs them to
${CMAKE_INSTALL_LIBDIR}/cmake/liblzma (I omitted the optional version
number). The path matches the CMake docs. I see quite a bit of stuff in
/usr/lib/cmake on my system. However, the path doesn't match what was
in the vcpkg files you linked. The CMakeLists.txt there uses
share/liblzma instead.
- Should I care about this? Shouldn't vcpkg fix it if they use
a non-standard path?
- Should CMakeLists.txt have a separate variable for the install
directory of the .cmake files so it can be overriden by the
callers? If yes, what variable name?
Is it correct to use this?
set(THREADS_PREFER_PTHREAD_FLAG TRUE)
find_package(Threads REQUIRED)
The first version used include(FindThreads) instead of find_package but
I changed it for the second version.
Are the generated .cmake files correctly done?
- I suppose liblzmaConfigVersion.cmake is OK. I know that
"COMPATIBILITY SameMajorVersion" is correct for liblzma.
- liblzmaConfig.cmake checks for the dependency on Threads::Threads.
Is it needed? Is it correctly done?
- What is a good namespace for liblzma? Doesn't liblzma::liblzma
conflict with FindLibLZMA since CMake is case insensitive?
Actually, doesn't the whole liblzma name conflict with FindLibLZMA
if FindLibLZMA is meant to be used via find_package(LibLZMA)?
- Can you test if the generated .cmake files work in practice?
I haven't tested them in any way.
CMakeLists.txt in vcpkg sets CMAKE_DEBUG_POSTFIX to "d". I hope I don't
need to do that. Isn't it caller's job to set such variables?
Would it be useful to specify "COMPONENT Release" and "COMPONENT
Development" in the install commands? To do it properly for GNU/Linux
requires NAMELINK_COMPONENT too which requires CMake 3.12.
Do you have any opinion how old CMake versions need to be supported?
If you wish to help with non-Windows support too, having proper checks
for the defines under "These are specific to GNU/Linux" w
Re: [xz-devel] cmake support?
Dear Lasse, awesome! Great work! On 17.02.20 01:54, Lasse Collin wrote: >>> Is there a need for liblzma-only CMake support outside Windows? >> >> I guess one main usage would be in the cases where another project >> uses CMake and pulls liblzma as a dependency and builds it in >> situations where a system provided liblzma is not available. It would >> be easier for those people to just use external_project without a lot >> of configuration with native CMake support instead of worrying about >> if bash or sh is available for configure > > Makes sense. Even if Windows might be the only CMake-supported OS that > lacks the capability to easily run configure, it's much simpler to stay > within the same build system type. Perhaps this can be considered > later. I think the conan community also enjoyes cmake because it integrates very well. So, since this CMakeLists.txt is quite well developed its not unlikely that more people looking mostly for liblzma will try to switch to cmake. Are you still considering this mostly for MSVC? If so, it may be helpful to add an explicit check, to avoid people trying to track down problems of missing 'config.h'. But if more platforms could qualify it would certainly be appreciated! Maybe some of the major platforms like Linux and macOS? Anything I can do to help? All the best, Mario -- BioDataAnalysis GmbH, Mario Emmenlauer Tel. Buero: +49-89-74677203 Balanstr. 43 mailto: memmenlauer * biodataanalysis.de D-81669 München http://www.biodataanalysis.de/
Re: [xz-devel] cmake support?
On 2020-02-16 Christopher Degawa wrote: > > but I don't have much clue how old CMake should be supported. > > > If you don't care about centos 7 using 2.8.12, 3.5 is a good lower > end to start with since that's what ubuntu 16.04 has in their default > repository. Any higher and you might find llvm-dev's discussion of > updating the cmake requirements to 3.15 of interest > http://lists.llvm.org/pipermail/llvm-dev/2019-October/136295.html > > (Centos 8 has 3.11 it seems, nice) Thanks! I will keep those version numbers in mind. I guess the importance of supporting old versions largely depends on if the CMake support will become usable for more than building liblzma on Windows. > On Sun, Feb 16, 2020 at 4:48 PM Lasse Collin > wrote: > > If I use add_library(lzma ...) and PREFIX "lib", isn't the end > > result practically identical to the current method? Or is it more > > about coding style and "liblzma" looks weird? > > > This was just more of a coding style since not many people usually > care to force a lib prefix on msvc built dlls. > In my opinion, I would say that forcing lib prefix is a bit weird, OK. I know it's a bit weird but I also think the alternative is worse. > but it might be fine in lzma's case since even CMake uses libLZMA > https://cmake.org/cmake/help/latest/module/FindLibLZMA.html Hmm... its source code is baffling. Sounds like they used LZMA SDK and decided to move to liblzma without changing the CMake module name, which is weird because for some tasks LZMA SDK is the best or even the only choice. This also makes me wonder if I should use xz::liblzma to avoid a case-insensitive namespace conflict with FindLibLZMA. liblzma has always installed a pkg-config file and CMake can use them too. It's unfortunate that it wasn't enough. Perhaps it's due to Windows again where liblzma.pc is missing. On the other hand, there's no #definition of LZMA_API_STATIC for static linking in FindLibLZMA so Windows support doesn't fully explain it either. > > Is there a need for liblzma-only CMake support outside Windows? > > I guess one main usage would be in the cases where another project > uses CMake and pulls liblzma as a dependency and builds it in > situations where a system provided liblzma is not available. It would > be easier for those people to just use external_project without a lot > of configuration with native CMake support instead of worrying about > if bash or sh is available for configure Makes sense. Even if Windows might be the only CMake-supported OS that lacks the capability to easily run configure, it's much simpler to stay within the same build system type. Perhaps this can be considered later. -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] cmake support?
On Sun, Feb 16, 2020 at 4:48 PM Lasse Collin wrote: > > - The lib prefix in the target generated by the add_library command > > is not needed as cmake will automatically prepend lib on appropriate > > targets, if you want to force it to always output as `liblzma` you > > could use > > > https://cmake.org/cmake/help/v3.1/variable/CMAKE_STATIC_LIBRARY_PREFIX.html > > or > > > https://cmake.org/cmake/help/v3.0/variable/CMAKE_SHARED_LIBRARY_PREFIX.html > > and set them to "lib" or set the prefix in line 269 instead of blank > > If I use add_library(lzma ...) and PREFIX "lib", isn't the end result > practically identical to the current method? Or is it more about coding > style and "liblzma" looks weird? > This was just more of a coding style since not many people usually care to force a lib prefix on msvc built dlls. In my opinion, I would say that forcing lib prefix is a bit weird, but it might be fine in lzma's case since even CMake uses libLZMA https://cmake.org/cmake/help/latest/module/FindLibLZMA.html > > - I'm not sure if `-Wl,--version-script` is supported on all OSs. > > It's not. configure enables the version script only on GNU/* and > FreeBSD. In CMakeLists.txt I only cared about Windows and GNU/Linux for > now. Even the GNU/Linux side is barely half done as it just assumes > things that are true only if the compiler and glibc are new enough > (e.g. clock_gettime() depends on librt in old glibc versions). > > Is there a need for liblzma-only CMake support outside Windows? > That's an ehh point, personally, if I see a CMakeLists in a project, I would try cmake before trying Autotools since a lot of times, Autotools files are written with linux in mind with not much consideration for other platforms until someone else adds support etc, although the Autotools configuration being broken is rare. I guess one main usage would be in the cases where another project uses CMake and pulls liblzma as a dependency and builds it in situations where a system provided liblzma is not available. It would be easier for those people to just use external_project without a lot of configuration with native CMake support instead of worrying about if bash or sh is available for configure Thanks for your help! > You're welcome
Re: [xz-devel] cmake support?
> > but I don't have much clue how old CMake should be supported. > If you don't care about centos 7 using 2.8.12, 3.5 is a good lower end to start with since that's what ubuntu 16.04 has in their default repository. Any higher and you might find llvm-dev's discussion of updating the cmake requirements to 3.15 of interest http://lists.llvm.org/pipermail/llvm-dev/2019-October/136295.html (Centos 8 has 3.11 it seems, nice)
Re: [xz-devel] cmake support?
On 2020-02-16 Lasse Collin wrote:
> I quickly checked a few random things and didn't find anything else
> that wouldn't be OK with 3.1.
target_link_options needs 3.13. Seems that it can be avoided by using
set_target_properties(liblzma PROPERTIES LINK_FLAGS
-Wl,--version-script=${CMAKE_CURRENT_LIST_DIR}/src/liblzma/liblzma.map
)
but I don't have much clue how old CMake should be supported.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
Re: [xz-devel] cmake support?
On 2020-02-16 Christopher Degawa wrote:
> I'm not a xz developer, but more of a user, I hope these can be of
> use to you
Thanks! I'm sure you have more experience about CMake than I do. :-)
> - Although you mark your cmake minimum as 3.1, your project command
> contains `DESCRIPTION`, and `HOMPAGE_URL`, both of which are not
> provided in cmake 3.1
HOMEPAGE_URL would need 3.12. I removed them.
I quickly checked a few random things and didn't find anything else
that wouldn't be OK with 3.1. 3.0 isn't OK because it lacks
THREADS_PREFER_PTHREAD_FLAG support in Threads (FindThreads).
> - The ${CMAKE_CURRENT_LIST_DIR} in your add_library are a bit
> unnecessary since any non-full paths are already assumed to be
> relative for add_library.
Fixed. I also fixed target_include_directories and target_sources.
Seems that the references to liblzma.map need ${CMAKE_CURRENT_LIST_DIR}
as a prefix. It makes sense though as CMake cannot know the context in
those cases.
> - The lib prefix in the target generated by the add_library command
> is not needed as cmake will automatically prepend lib on appropriate
> targets, if you want to force it to always output as `liblzma` you
> could use
> https://cmake.org/cmake/help/v3.1/variable/CMAKE_STATIC_LIBRARY_PREFIX.html
> or
> https://cmake.org/cmake/help/v3.0/variable/CMAKE_SHARED_LIBRARY_PREFIX.html
> and set them to "lib" or set the prefix in line 269 instead of blank
If I use add_library(lzma ...) and PREFIX "lib", isn't the end result
practically identical to the current method? Or is it more about coding
style and "liblzma" looks weird?
I want to avoid the name lzma.dll because it would be a namespace
conflict with LZMA SDK. I assume there is no "lib" prefix on Windows by
default. The MinGW-w64 builds of XZ Utils have liblzma.dll and I suppose
it would be good to keep the same name with other builds too.
> - for line 214 when building with `cmake ${XZ_SOURCE}
> -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Debug`
> ```
> CMake Warning (dev) at CMakeLists.txt:42 (add_library):
> Policy CMP0063 is not set: Honor visibility properties for all
> target types.
Hopefully fixed.
> - I'm not sure if `-Wl,--version-script` is supported on all OSs.
It's not. configure enables the version script only on GNU/* and
FreeBSD. In CMakeLists.txt I only cared about Windows and GNU/Linux for
now. Even the GNU/Linux side is barely half done as it just assumes
things that are true only if the compiler and glibc are new enough
(e.g. clock_gettime() depends on librt in old glibc versions).
Is there a need for liblzma-only CMake support outside Windows?
I changed a few other things too. Instead of include(FindThreads) it now
uses find_package(Threads REQUIRED) so it catches if threading support
is missing (adding CMake support for unthreaded build isn't worth it,
I guess). I also added find_dependency(Threads) to liblzmaConfig.cmake
since I got an impression from other sources that it is required.
Thanks for your help!
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
#
#
# This provides very limited CMake support:
# - Static or shared liblzma only (no command line tools or tests)
# - GNU/Linux with GCC or Clang, or Windows with MSVC
# - 32/64-bit x86 and 32/64-bit little endian ARM
#
# For now, this is indented to be useful on Windows. The GNU/Linux support
# is just for testing and shouldn't normally be used.
#
#
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#
cmake_minimum_required(VERSION 3.1)
# Get the package version from version.h into XZ_VERSION variable.
file(READ src/liblzma/api/lzma/version.h XZ_VERSION)
string(REGEX REPLACE
"^.*\n\
#define LZMA_VERSION_MAJOR ([0-9]+)\n\
#define LZMA_VERSION_MINOR ([0-9]+)\n\
#define LZMA_VERSION_PATCH ([0-9]+)\n\
.*$"
"\\1.\\2.\\3" XZ_VERSION ${XZ_VERSION})
# While this file only builds liblzma, name the project xz still in case
# some day this file builds more than liblzma. Among other things, this
# gives us variables xz_VERSION and xz_VERSION_MAJOR.
project(xz VERSION ${XZ_VERSION} LANGUAGES C)
add_library(liblzma
src/common/tuklib_cpucores.c
src/common/tuklib_physmem.c
src/liblzma/check/check.c
src/liblzma/check/crc32_fast.c
src/liblzma/check/crc32_table.c
src/liblzma/check/crc64_fast.c
src/liblzma/check/crc64_table.c
src/liblzma/check/sha256.c
src/liblzma/common/alone_decoder.c
src/liblzma/common/alone_encoder.c
src/liblzma/common/auto_decoder.c
src/liblzma/common/block_buffer_decoder.c
src/liblzma/common/block_buffer_encoder.c
src/liblzma/common/block_decoder.c
src/liblzma/common/block_encoder.c
src/liblzma/common/block_header_decoder.c
src
Re: [xz-devel] cmake support?
I'm not a xz developer, but more of a user, I hope these can be of use to
you
- Although you mark your cmake minimum as 3.1, your project command
contains `DESCRIPTION`, and `HOMPAGE_URL`, both of which are not provided
in cmake 3.1 https://cmake.org/cmake/help/v3.1/command/project.html I did
not check for any other incompatibilities.
- The ${CMAKE_CURRENT_LIST_DIR} in your add_library are a bit
unnecessary since any non-full paths are already assumed to be relative for
add_library.
- The lib prefix in the target generated by the add_library command is not
needed as cmake will automatically prepend lib on appropriate targets, if
you want to force it to always output as `liblzma` you could use
https://cmake.org/cmake/help/v3.1/variable/CMAKE_STATIC_LIBRARY_PREFIX.html
or
https://cmake.org/cmake/help/v3.0/variable/CMAKE_SHARED_LIBRARY_PREFIX.html and
set them to "lib" or set the prefix in line 269 instead of blank
- for line 214 when building with `cmake ${XZ_SOURCE}
-DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Debug`
```
CMake Warning (dev) at CMakeLists.txt:42 (add_library):
Policy CMP0063 is not set: Honor visibility properties for all target
types. Run "cmake --help-policy CMP0063" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Target "liblzma" of type "STATIC_LIBRARY" has the following visibility
properties set for C:
C_VISIBILITY_PRESET
For compatibility CMake is not honoring them for this target.
This warning is for project developers. Use -Wno-dev to suppress it.
```
https://cmake.org/cmake/help/v3.3/policy/CMP0063.html
You can ignore it without any problems, it just may look ugly for an
end-user trying to compile using cmake
- I'm not sure if `-Wl,--version-script` is supported on all OSs. I think
AppleClang doesn't, but it's probably fine to wait until a macOS user can
provide info or not
On Sun, Feb 16, 2020 at 11:12 AM Lasse Collin
wrote:
> On 2020-02-15 Mario Emmenlauer wrote:
> > This has been my foremost reason for cmake. While not everything is
> > perfect, it has proven to work very well for generating native build
> > instructions with well defined settings on quite a number of
> > platforms.
>
> CMake can be used like GNU Autotools to make portable builds but in this
> case it's not practical to move the whole project to CMake. It should
> be enough to have something that does as little as possible to produce
> usable output for MSVC and ignore the portability features of CMake...
>
> > I would start with the instructions from vcpkg (see
> > https://github.com/microsoft/vcpkg/tree/master/ports/liblzma) but
> > unify their multiple patches into a single, concise CMakeLists.txt.
>
> ...and these files seem to do that, and they only build liblzma. Thanks!
>
> There are several details that I don't understand (yet). I see some
> small minor problems though. liblzma_w32res.rc is missing from from
> Windows DLL output. The "NOT WIN32" part looks a bit broken even though
> it might compile. For example:
>
> - The definition of SIZEOF_SIZE_T is wrong as it's not understandable
> by the preprocessor but it's often not needed either so a compiler
> may not care.
>
> - Some #defines are missing that are needed for a good build on
> GNU/Linux.
>
> I wonder why enable-uwp-builds.patch is needed. Is GetModuleHandle a
> problem on UWP? If so, it can be changed to make things simpler.
> Compatibility with Windows 98 isn't that important... ;-)
>
> Since I would be maintaining the CMakeLists.txt file, I thought I should
> understand it too. After reading some CMake docs and the linked vcpkg
> files I ended up with the attached file. It does a little more than
> just MSVC support so I can test it on GNU/Linux too. It only works
> against xz.git, but if this can be finished quickly, a version can be
> included in the upcoming XZ Utils 5.2.5.
>
> Some known issues with my CMakeLists.txt:
>
> - The Windows parts aren't tested at all. The file header says
> 32/64-bit x86/ARM but I tried only x86-64 GNU/Linux.
>
> - It obviously doesn't attempt to be properly portable since it
> assumes little endian etc. but that isn't the goal for now either.
>
> - I don't have much clue about the generated .cmake files. E.g.
> is the single-line liblzmaConfig.cmake fine or does it need
> something extra to handle the dependency on Threads::Theads?
>
> - The default installation paths come from GNUInstallDirs.
> The .cmake files are installed into a different directory
> than in the vcpkg files.
>
> - All #defines are passed as command line arguments. That is, it
> doesn't create or use config.h. The number of #defines isn't
> huge so it the command line lengths don't get too long, but
> is it bad/ugly on Windows?
>
> - NDEBUG shouldn't be #defined for debug builds.
>
> Thoughts, fixes, suggestions etc. are welcome.
>
> --
> Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
>
Re: [xz-devel] cmake support?
On 2020-02-15 Mario Emmenlauer wrote:
> This has been my foremost reason for cmake. While not everything is
> perfect, it has proven to work very well for generating native build
> instructions with well defined settings on quite a number of
> platforms.
CMake can be used like GNU Autotools to make portable builds but in this
case it's not practical to move the whole project to CMake. It should
be enough to have something that does as little as possible to produce
usable output for MSVC and ignore the portability features of CMake...
> I would start with the instructions from vcpkg (see
> https://github.com/microsoft/vcpkg/tree/master/ports/liblzma) but
> unify their multiple patches into a single, concise CMakeLists.txt.
...and these files seem to do that, and they only build liblzma. Thanks!
There are several details that I don't understand (yet). I see some
small minor problems though. liblzma_w32res.rc is missing from from
Windows DLL output. The "NOT WIN32" part looks a bit broken even though
it might compile. For example:
- The definition of SIZEOF_SIZE_T is wrong as it's not understandable
by the preprocessor but it's often not needed either so a compiler
may not care.
- Some #defines are missing that are needed for a good build on
GNU/Linux.
I wonder why enable-uwp-builds.patch is needed. Is GetModuleHandle a
problem on UWP? If so, it can be changed to make things simpler.
Compatibility with Windows 98 isn't that important... ;-)
Since I would be maintaining the CMakeLists.txt file, I thought I should
understand it too. After reading some CMake docs and the linked vcpkg
files I ended up with the attached file. It does a little more than
just MSVC support so I can test it on GNU/Linux too. It only works
against xz.git, but if this can be finished quickly, a version can be
included in the upcoming XZ Utils 5.2.5.
Some known issues with my CMakeLists.txt:
- The Windows parts aren't tested at all. The file header says
32/64-bit x86/ARM but I tried only x86-64 GNU/Linux.
- It obviously doesn't attempt to be properly portable since it
assumes little endian etc. but that isn't the goal for now either.
- I don't have much clue about the generated .cmake files. E.g.
is the single-line liblzmaConfig.cmake fine or does it need
something extra to handle the dependency on Threads::Theads?
- The default installation paths come from GNUInstallDirs.
The .cmake files are installed into a different directory
than in the vcpkg files.
- All #defines are passed as command line arguments. That is, it
doesn't create or use config.h. The number of #defines isn't
huge so it the command line lengths don't get too long, but
is it bad/ugly on Windows?
- NDEBUG shouldn't be #defined for debug builds.
Thoughts, fixes, suggestions etc. are welcome.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
#
#
# This provides very limited CMake support:
# - Static or shared liblzma only (no command line tools or tests)
# - GNU/Linux with GCC or Clang, or Windows with MSVC
# - 32/64-bit x86 and 32/64-bit little endian ARM
#
# For now, this is indented to be useful on Windows. The GNU/Linux support
# is just for testing and shouldn't normally be used.
#
#
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#
cmake_minimum_required(VERSION 3.1)
# Get the package version from version.h into XZ_VERSION variable.
file(READ ${CMAKE_CURRENT_LIST_DIR}/src/liblzma/api/lzma/version.h XZ_VERSION)
string(REGEX REPLACE
"^.*\n\
#define LZMA_VERSION_MAJOR ([0-9]+)\n\
#define LZMA_VERSION_MINOR ([0-9]+)\n\
#define LZMA_VERSION_PATCH ([0-9]+)\n\
.*$"
"\\1.\\2.\\3" XZ_VERSION ${XZ_VERSION})
# While this file only builds liblzma, name the project xz still in case
# some day this file builds more than liblzma. Among other things, this
# gives us variables xz_VERSION, xz_VERSION_MAJOR, and xz_HOMEPAGE_URL.
project(xz
VERSION ${XZ_VERSION}
DESCRIPTION "XZ Utils (liblzma only)"
HOMEPAGE_URL "https://tukaani.org/xz/";
LANGUAGES C
)
add_library(liblzma
${CMAKE_CURRENT_LIST_DIR}/src/common/tuklib_cpucores.c
${CMAKE_CURRENT_LIST_DIR}/src/common/tuklib_physmem.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/check/check.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/check/crc32_fast.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/check/crc32_table.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/check/crc64_fast.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/check/crc64_table.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/check/sha256.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/common/alone_decoder.c
${CMAKE_CURRENT_LIST_DIR}/src/liblzma/common/alone_
Re: [xz-devel] cmake support?
Dear Lasse, On 15.02.20 15:41, Lasse Collin wrote: On 2020-02-15 Mario Emmenlauer wrote: Would you consider integrating cmake support? Maybe. I remember this being asked on IRC years ago but I wasn't able to think about it back then. If it is enough to have CMake support to build liblzma on a few very specific targets, it shouldn't be too hard to add it *and* keep it maintained. Supporting more than liblzma (like the xz tool) and many operating systems would be too much work. Such a full support would require porting the various Autoconf tests to CMake, testing the tests, and keeping them up to date when the Autoconf tests are updated. I think that is a perfectly reasonable approach. It will also give you time to get acquainted with cmake at a low entry cost, while providing hopefully significant benefit to users with less common build configurations. My reason for asking is that I've been using unofficial cmake patches since a while to build xz on MSVC, and its always difficult to build trust into every new release. There are Visual Studio project files in XZ Utils to build liblzma. Are those project files problematic or not good enough? Admittedly I did not use them, because I'm very keen on using the same build configuration (with respect to hardware optimizations and compiler settings) in my full build stack. It seemed easier to re-generate the project files than to validate what options they set (albeit I may be considered a bit pedantic there). Even if they are fine, I admit it could be nice if CMake could be used instead, since maintaining the VS project files is a bit annoying and I don't even test them. This has been my foremost reason for cmake. While not everything is perfect, it has proven to work very well for generating native build instructions with well defined settings on quite a number of platforms. For what its worth there would be existing cmake integration that can be used as a head start, i.e. from vcpkg. A link to a known good CMakeLists.txt (and possible other files) would be nice along with an explanation what it builds (e.g. both static and shared liblzma?) and which targets are supported (e.g. is it MSVC only?). Thanks! If you consider this an option, I could make a proposal in a feature branch. I would start with the instructions from vcpkg (see https://github.com/microsoft/vcpkg/tree/master/ports/liblzma) but unify their multiple patches into a single, concise CMakeLists.txt. Please don't get discouraged when you browse their source - its just split into patches that would not necessarily be required. One more question: I would not start with an auto-generated header for config.h but follow a similar approach to the current win32 headers. Is that acceptable for a start? All the best, Mario Emmenlauer
Re: [xz-devel] cmake support?
On 2020-02-15 Mario Emmenlauer wrote: > Would you consider integrating cmake support? Maybe. I remember this being asked on IRC years ago but I wasn't able to think about it back then. If it is enough to have CMake support to build liblzma on a few very specific targets, it shouldn't be too hard to add it *and* keep it maintained. Supporting more than liblzma (like the xz tool) and many operating systems would be too much work. Such a full support would require porting the various Autoconf tests to CMake, testing the tests, and keeping them up to date when the Autoconf tests are updated. > My reason for asking is that I've been using unofficial cmake patches > since a while to build xz on MSVC, and its always difficult to build > trust into every new release. There are Visual Studio project files in XZ Utils to build liblzma. Are those project files problematic or not good enough? Even if they are fine, I admit it could be nice if CMake could be used instead, since maintaining the VS project files is a bit annoying and I don't even test them. > For what its worth there would be existing cmake integration that > can be used as a head start, i.e. from vcpkg. A link to a known good CMakeLists.txt (and possible other files) would be nice along with an explanation what it builds (e.g. both static and shared liblzma?) and which targets are supported (e.g. is it MSVC only?). Thanks! -- Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
