Hi Haonan, Thanks for this information … I was just about to say: If you copy stuff in the repo, you must list it in the LICENSES file together with which license it’s under and bundle the license text along with the rest.
If it’s really just a hand full of files from a stable repo with a license that allows integration, and pulling in the full third party repo is total overkill, then I’d agree to integrate these. If it’s just „simpler“, I’d prefer the other approaches. Chris Von: Haonan Hou <[email protected]> Datum: Donnerstag, 9. Juli 2026 um 10:51 An: [email protected] <[email protected]> Betreff: Re: Re:AW: Proposal for Managing Third-Party Source Code in the C++ Module Hi, I looked a bit at how Apache Arrow C++ handles third-party dependencies, and I think it may be a useful reference for this discussion. Arrow does not simply put all third-party source code directly into the main repository. Instead, its CMake build supports several dependency resolution modes through `ARROW_DEPENDENCY_SOURCE`, such as `AUTO`, `SYSTEM`, `BUNDLED`, `CONDA`, `VCPKG`, and `BREW`. In particular, the `BUNDLED` mode usually means that Arrow builds the dependency from a pinned source archive during the build, rather than maintaining the full upstream source tree in the repository. The dependency versions, source URLs, and SHA256 checksums are recorded in `cpp/thirdparty/versions.txt`. Arrow also provides `cpp/thirdparty/download_dependencies.sh` to pre-download these archives for offline builds. At the same time, Arrow does keep a small number of vendored source files in-tree under paths such as `cpp/src/arrow/vendored/`. These cases are explicitly documented in `LICENSE.txt`, including the upstream source and license information. So Arrow seems to use a mixed approach: - Prefer system or environment-provided dependencies when suitable. - Allow bundled builds from pinned source archives for portability and reproducibility. - Support offline builds by pre-downloading dependency archives. - Keep only selected vendored source code in-tree, with clear license documentation. - Avoid treating in-repository third-party source copies as the default dependency management model. For TsFile C++, maybe we can consider a similar compromise. Instead of always copying third-party source code into `cpp/third_party`, we could introduce a dependency policy like: - Use system dependencies where reasonable. - Provide a bundled build option for important C++ dependencies, using pinned versions, URLs, and checksums. - Support offline builds by allowing users to pre-download dependency archives. - Only vendor source code directly in the repository for small, stable, carefully reviewed cases. - For any in-tree vendored code, require clear documentation of origin, version or commit, license, trimming scope, and local modifications. This may address the portability and build stability concerns while also reducing legal, licensing, and long-term maintenance risks. References: - Arrow C++ dependency management: https://arrow.apache.org/docs/developers/cpp/building.html#build-dependency-management - Arrow bundled dependency versions: https://github.com/apache/arrow/blob/main/cpp/thirdparty/versions.txt - Arrow license documentation for vendored/included code: https://github.com/apache/arrow/blob/main/LICENSE.txt Best, Haonan On 2026/07/09 07:42:52 ColinLee wrote: > Hi Chris, > > > Thanks for raising this. Your concerns make sense, especially around long-term > maintenance, redistribution rights, and the risk of treating vendored source > as > a default dependency-management pattern. > > > Let me clarify the scope of what I had in mind. The third-party code I was > looking at is mainly for encoding, decoding, and compression support. These > implementations are generally mature and stable, and the corresponding file > formats or compression formats do not change frequently. > > > I also agree that copying third-party code into the repository should not > become > a general dependency-management approach. For dependencies that are actively > evolving, security-sensitive, or likely to require frequent upstream > synchronization, we should discuss the appropriate integration approach > case by case before introducing them, rather than defaulting to vendoring the > source code. > > > My intent was only to explore whether a small set of stable codec and > compression implementations might need a different treatment for C++ build > portability and format compatibility. I look forward to further discussion on > the legal, licensing, maintenance, and build-portability aspects so that we > can > choose the right approach together. > > > Best, > Colin > > > > > > > > > > > > > > At 2026-07-09 14:42:30, "Christofer Dutz" <[email protected]> wrote: > >Hi Colin, > > > >Are you proposing to copy the code of another project into the repo here and > >to also release that alongside the project? > > > >If that’s the case I would strongly suggest to not do that. > > > >In general, this would be a hostile fork of the source project and you end > >up having to manually sync it. > > > >The way more pressing problem is that we generally only have code in our > >repositories and releases that we have the rights to share. We wouldn’t have > >this from the original project. > > > >I don’t know the specifics of the project you want to bring in, but I think > >copying it here will cause more legal, license and trademark issues than it > >will solve build issues. > > > >Chris > > > > > >Von: ColinLee <[email protected]> > >Datum: Donnerstag, 9. Juli 2026 um 05:04 > >An: dev <[email protected]> > >Betreff: Proposal for Managing Third-Party Source Code in the C++ Module > > > > > >Hi all, > > > >To improve build stability across different platforms and usage scenarios, I > >propose that we manage necessary third-party libraries by maintaining their > >source code directly in the repository. > > > >## Why We Vendor Source Code > > > >The C++ module needs to support Linux, macOS, MSVC, embedded environments, > >and other build environments. If we rely on system libraries or download > >dependencies during the build, it becomes harder to control library > >versions, compiler options, and network availability. > > > >By keeping the source code under `cpp/third_party`, we can better control > >build details such as PIC, static linking, MSVC runtime settings, and source > >trimming. It also helps reduce the extra dependency installation burden for > >users of `libtsfile`. > > > >## Initial Import and Commit Structure > > > >Third-party source code should be placed under > >`cpp/third_party/<third-party-dir>/`. We should only keep the source subset > >that is actually needed, and preserve the corresponding license files. > > > >I also suggest adding a README to record the source origin, version or > >upstream commit, trimming scope, and license information. > > > >Suggested commit structure: > > > >- First commit: import the third-party source code, license files, and > >README. > >- Second commit: integrate it with TsFile logic and local CMake glue. > > > >## Future Changes > > > >In general, we should avoid modifying third-party source code directly. > >Prefer CMake options, wrappers, or adapters when possible. > > > >If third-party source code must be modified, the change should be kept > >separate from TsFile logic as much as possible. Alternatively, the > >corresponding commit should include a patch that tracks the local > >modification against upstream. > > > >Future commits should also remain reasonably independent: > > > >- Third-party version upgrade: separate commit. > >- Third-party source modification or patch: separate commit when possible, > >or include the patch in the corresponding commit. > > > >This keeps third-party code origins clear, licenses traceable, and local > >changes reviewable. It also makes future upgrades or rollbacks easier. > > > >Thanks. > > > >colin >
