i18nsite opened a new pull request, #3255:
URL: https://github.com/apache/kvrocks/pull/3255
# build: Optimize CMake for better compatibility with Nix/NixOS build system
| 优化 CMake 以更好地兼容 Nix/NixOS 构建系统
---
## English
### What is Nix/NixOS?
**Nix** is a purely functional package manager that ensures reproducible
builds and deployments. **NixOS** is a Linux distribution built on top of Nix.
Key characteristics of the Nix build system:
- **Read-only source directories**: All source code is stored in
`/nix/store` with read-only permissions
- **Isolated build environments**: Each build runs in a sandboxed
environment with only declared dependencies
- **Reproducible builds**: Same inputs always produce the same outputs
- **Pre-built dependencies**: Dependencies are built separately and cached,
then provided to dependent packages
### The Problem
Kvrocks' current CMake configuration uses `FetchContent` to download and
build several dependencies (jemalloc, lua, luajit, lz4, zstd) from source
during the build process. This approach has issues in Nix environments:
1. **Read-only source directories**: The build process tries to write to
source directories in `/nix/store`, which fails due to permission errors
2. **Autoconf/configure issues**: Tools like `autoconf` and `configure` need
to modify source files, which is impossible in read-only directories
3. **Build inefficiency**: Dependencies are rebuilt for every package
instead of being cached and reused
Without this change, building Kvrocks in Nix requires:
- Complex workarounds to copy source directories to writable locations
- Patching multiple CMake files extensively
- Maintaining separate build logic for Nix
### The Solution
This PR modifies 5 CMake files to check if dependency libraries already
exist before attempting to build them:
**Modified files:**
- `cmake/jemalloc.cmake`
- `cmake/lua.cmake`
- `cmake/luajit.cmake`
- `cmake/lz4.cmake`
- `cmake/zstd.cmake`
**Changes made:**
```cmake
# Before: Always build from source
add_custom_target(make_xxx COMMAND ${MAKE_COMMAND} ...
WORKING_DIRECTORY ...
BYPRODUCTS ...
)
# After: Check if pre-built library exists
if(NOT EXISTS ${xxx_SOURCE_DIR}/lib/libxxx.a)
# Build from source if not exists
add_custom_target(make_xxx COMMAND ${MAKE_COMMAND} ...
WORKING_DIRECTORY ...
BYPRODUCTS ...
)
else()
# Use pre-built library, create empty target
add_custom_target(make_xxx)
endif()
```
### How It Works
1. During CMake configuration, check if the library file exists
2. If the file doesn't exist: proceed with normal build process (no change
in behavior)
3. If the file exists: create an empty custom target to maintain dependency
relationships
4. Build systems like Nix can place pre-built libraries in expected
locations before the build phase
### Benefits
✅ **Zero impact on normal builds**: Regular builds continue to work exactly
as before
✅ **Enables Nix support**: Makes it possible to package Kvrocks for NixOS
✅ **Faster builds**: Pre-built and cached dependencies significantly reduce
build time
✅ **Better reproducibility**: Pre-built dependencies ensure consistent
builds across environments
✅ **Minimal changes**: Only ~30 lines changed across 5 files
### Testing
- ✅ **Docker build**: Verified normal build path works correctly
- ✅ **Nix build**: Successfully built on macOS (Apple Silicon) with Nix
- ✅ **Binary verification**: `kvrocks --version` works as expected
- ✅ **All dependencies**: jemalloc, lua, luajit, lz4, zstd pre-built and
linked correctly
### Use Cases
This change benefits:
- **NixOS users**: Can now install Kvrocks through the Nix package manager
- **Reproducible builds**: CI/CD pipelines with cached dependencies
- **Development environments**: Faster iteration with pre-built dependencies
- **Sandboxed builds**: Any build system with read-only source requirements
---
## 中文
### 什么是 Nix/NixOS？
**Nix** 是一个纯函数式包管理器，确保可重现的构建和部署。**NixOS** 是基于 Nix 构建的 Linux 发行版。
Nix 构建系统的关键特性：
- **只读源码目录**：所有源代码存储在 `/nix/store` 中，具有只读权限
- **隔离的构建环境**：每个构建在沙盒环境中运行，只能访问声明的依赖
- **可重现构建**：相同的输入始终产生相同的输出
- **预编译依赖**：依赖项单独构建并缓存，然后提供给依赖包
### 问题描述
Kvrocks 当前的 CMake 配置使用 `FetchContent`
在构建过程中从源码下载并编译多个依赖项（jemalloc、lua、luajit、lz4、zstd）。这种方法在 Nix 环境中存在问题：
1. **只读源码目录**：构建过程尝试写入 `/nix/store` 中的源码目录，由于权限错误而失败
2. **Autoconf/configure 问题**：`autoconf` 和 `configure` 等工具需要修改源文件，在只读目录中无法实现
3. **构建效率低下**：每个包都重新构建依赖项，而不是缓存和重用
如果不做这些改动，在 Nix 中构建 Kvrocks 需要：
- 复杂的变通方法将源码目录复制到可写位置
- 大量修改多个 CMake 文件
- 为 Nix 维护单独的构建逻辑
### 解决方案
本 PR 修改了 5 个 CMake 文件，在尝试构建依赖库之前检查它们是否已经存在：
**修改的文件：**
- `cmake/jemalloc.cmake`
- `cmake/lua.cmake`
- `cmake/luajit.cmake`
- `cmake/lz4.cmake`
- `cmake/zstd.cmake`
**所做的改动：**
```cmake
# 之前：总是从源码构建
add_custom_target(make_xxx COMMAND ${MAKE_COMMAND} ...
WORKING_DIRECTORY ...
BYPRODUCTS ...
)
# 之后：检查预编译库是否存在
if(NOT EXISTS ${xxx_SOURCE_DIR}/lib/libxxx.a)
# 如果不存在则从源码构建
add_custom_target(make_xxx COMMAND ${MAKE_COMMAND} ...
WORKING_DIRECTORY ...
BYPRODUCTS ...
)
else()
# 使用预编译库，创建空目标
add_custom_target(make_xxx)
endif()
```
### 工作原理
1. 在 CMake 配置期间，检查库文件是否存在
2. 如果文件不存在：继续正常的构建过程（行为无变化）
3. 如果文件存在：创建一个空的自定义目标以维持依赖关系
4. 像 Nix 这样的构建系统可以在构建阶段之前将预编译库放置在预期位置
### 优势
✅ **对正常构建零影响**：常规构建继续完全按照之前的方式工作
✅ **启用 Nix 支持**：使得为 NixOS 打包 Kvrocks 成为可能
✅ **更快的构建**：预编译和缓存的依赖项显著减少构建时间
✅ **更好的可重现性**：预编译依赖确保跨环境的一致构建
✅ **最小改动**：仅在 5 个文件中修改了约 30 行
### 测试
- ✅ **Docker 构建**：验证正常构建路径正确工作
- ✅ **Nix 构建**：在 macOS (Apple Silicon) 上使用 Nix 成功构建
- ✅ **二进制验证**：`kvrocks --version` 按预期工作
- ✅ **所有依赖**：jemalloc、lua、luajit、lz4、zstd 正确预编译和链接
### 使用场景
此改动有益于：
- **NixOS 用户**：现在可以通过 Nix 包管理器安装 Kvrocks
- **可重现构建**：具有缓存依赖的 CI/CD 流水线
- **开发环境**：使用预编译依赖更快地迭代
- **沙盒构建**：任何具有只读源码要求的构建系统
---
## Labels
- `cmake`
- `build`
- `nix`
- `nixos`
- `compatibility`
## Checklist
- [x] Code follows the project's coding style
- [x] Changes are minimal and focused
- [x] No breaking changes to existing builds
- [x] Tested with Docker (normal build)
- [x] Tested with Nix build system
- [x] Binary runs correctly
- [x] Commit message follows conventional commits format
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
