farmerx opened a new pull request, #14005: URL: https://github.com/apache/apisix/pull/14005
### Description <!-- Please include a summary of the change and which issue is fixed. --> <!-- Please also include relevant motivation and context. --> The `tencent-cloud-cls` plugin always uploads logs as an uncompressed protobuf payload, although the CLS structured log API accepts compressed uploads through the `x-cls-compress-type` request header (see [上传结构化日志](https://cloud.tencent.com/document/product/614/16873)). Compressing the payload reduces the log write traffic remarkably (about 21x on realistic access logs), which is the main cost driver for high volume deployments. This PR adds optional zstd compression for the upload, mirroring the behaviour of the official Go SDK (`zstdCompress` + `x-cls-compress-type: zstd`). Changes: 1. New `apisix/utils/zstd.lua`: a thin LuaJIT FFI binding over libzstd (`ZSTD_compressBound`, `ZSTD_compress`, `ZSTD_isError`, `ZSTD_getErrorName`), with the default compression level 3 (`ZSTD_CLEVEL_DEFAULT`, the same as `zstd.SpeedDefault` used by the Go SDK). libzstd is loaded lazily and the result - including the failure - is cached, trying `libzstd.so.1`, `zstd` and `libzstd` in order so that environments shipping only the versioned soname work too. `available()` reports whether zstd is usable, `compress()` returns `nil, err` instead of throwing. 2. `apisix/plugins/tencent-cloud-cls/cls-sdk.lua`: `_M.new()` accepts an extra optional `compress_type` argument (unknown values are rejected); `send_cls_request()` compresses the encoded protobuf when `zstd` is configured and adds the `x-cls-compress-type: zstd` header. `Content-Type`, the signature and the request URL are unchanged. When compression fails, e.g. because libzstd is missing at runtime, the log is uploaded uncompressed and an error is logged - a missing native library never causes log loss. 3. `apisix/plugins/tencent-cloud-cls.lua`: new `compress_type` attribute (`"none"` | `"zstd"`, default `"none"`), passed through to the SDK. 4. Docs: `compress_type` and the libzstd requirement are documented in both `docs/en/latest/plugins/tencent-cloud-cls.md` and `docs/zh/latest/plugins/tencent-cloud-cls.md`. 5. `utils/install-dependencies.sh`: installs `libzstd-dev` (apt) and `libzstd-devel` (yum) so that dev/CI environments have libzstd available. Tests: - New `t/utils/zstd.t`: `available()`, argument validation, the zstd frame magic number `28 b5 2f fd`, the compressed size and compression levels 1/3/9 (the compression cases are skipped when libzstd is not installed). - `t/plugin/tencent-cloud-cls.t` TEST 23-26: the schema rejects unsupported compress types, a route can be created with `compress_type: zstd`, the uploaded body is a zstd frame and carries `x-cls-compress-type: zstd`, and compression failure falls back to an uncompressed upload that is still processed successfully. Notes for reviewers: this introduces a runtime dependency on libzstd (only when `compress_type: zstd` is used). Images that do not ship `libzstd1` should add it; without it the plugin keeps working and just uploads uncompressed payloads. #### Which issue(s) this PR fixes: <!-- *Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [x] I have added tests corresponding to this change - [x] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
