Hi all,

Following the previous discussion and vote on C++ third-party dependency 
management, 
the implementation is now ready for community review.

Discussion:
https://lists.apache.org/thread/1ssk1rot302cx88pkt3rvzxjr19kpppr

Vote and follow-up:
https://lists.apache.org/thread/ps3dxy236gnnn0rr38gfklbz5chf5cbo

Pull request:
https://github.com/apache/tsfile/pull/905

The main changes are:

1. Remove committed third-party source trees from cpp/third_party.

2. Introduce TSFILE_DEPENDENCY_SOURCE with three modes:

   - AUTO: prefer a compatible system dependency and otherwise use a verified 
upstream source archive.
   - SYSTEM: require compatible dependencies installed on the system and never 
download sources.
   - BUNDLED: download or reuse pinned upstream archives and build them in the 
CMake build directory.

   In this implementation, BUNDLED does not mean that third-party source code 
is committed to the TsFile repository.

3. Pin every downloaded dependency by version or commit and verify its SHA-256 
checksum before extraction.

4. Support offline builds through TSFILE_DEPENDENCY_OFFLINE and 
TSFILE_DEPENDENCY_CACHE.

5. Document dependency origins, versions, trimming scope, local modifications, 
and license information.

6. Migrate ANTLR4, utf8cpp, Snappy, LZ4, lzokay, SIMDe, zlib, Zstandard, and 
liblzma/XZ Utils to the new mechanism.

7. Add Zstandard and LZMA2 compression support, together with CHIMP, RLBE, and 
CAMEL encoding support.

8. Add CI coverage for SYSTEM, BUNDLED, offline, cross-platform, and Java/C++ 
compatibility builds.

All selected dependency licenses have been reviewed and documented. 

One area that may deserve particular attention is XZ Utils: the upstream 
archive contains files under different licenses, 
but the C++ build selects only the 0BSD-licensed liblzma source files. GPL/LGPL 
tools and components are not compiled, 
linked, installed, or redistributed.

Regarding the C++ language standard, the TsFile C++ implementation and public 
API remain C++11 compatible. 
The bundled lzokay source and some test targets require C++14. A strict C++11 
build is still available by disabling lzokay and tests. 
On MSVC, /std:c++14 is used because MSVC does not provide a /std:c++11 option.
The PR currently passes the Linux, macOS, MinGW, MSVC, Python, compatibility, 
dependency-source, offline, CodeQL, 
cppcheck, Spotless, and Apache RAT checks.

The PR has a large file count mainly because the previously committed 
third-party source trees are being removed.

It would be helpful if reviewers could pay particular attention to:

- The AUTO, SYSTEM, and BUNDLED behavior.
- Dependency versions, checksums, and download sources.
- License and NOTICE handling.
- The liblzma source and license boundary.
- CMake behavior across Linux, macOS, MinGW, and MSVC.
- The C++11 compatibility boundary.
- Cross-language codec compatibility.

Comments can be provided either in this thread or directly on PR #905.

Thanks,
Colin

Reply via email to