pjfanning opened a new pull request, #3710: URL: https://github.com/apache/pekko/pull/3710
### Motivation `ZstdDecompressor` creates a bare `ZstdDecompressCtx`. libzstd allocates the window buffer from the frame header before any decompressed-size limit applies, so a small input can make the decoder allocate up to 128 MiB per stream. Server-side users such as pekko-http's `zstd` content-coding need a lower limit; RFC 9659 caps it at a window log of 23 (8 MiB). zstd-jni's `ZstdDecompressCtx.setLongMax` (luben/zstd-jni#431) sets that limit, but no zstd-jni release includes it yet (the latest is 1.5.7-24). This PR adds the setting for 2.0.0, so finishing the work later only needs a dependency bump. ### Modification - `ZstdDecompressionSettings.withWindowLogMax(Int)` and `windowLogMax`. 0, the default, keeps libzstd's limit. This mirrors `ZstdCompressionSettings.withWindowLog`, and the validation range is the same. - `ZstdDecompressor` looks up `setLongMax` reflectively, because zstd-jni is an optional dependency that users add themselves. If the method is missing, a configured limit fails materialization with an `UnsupportedOperationException` rather than being silently ignored. The default path is unchanged. - Document the option, its zstd-jni requirement and the RFC 9659 limit on the `zstdDecompress` page. ### Result On 2.0.0, users can configure the window limit, and it takes effect as soon as they use a zstd-jni release with `setLongMax`. The follow-up (2.0.1) bumps `zstdJniVersion` and replaces the reflection with a direct call; there is a TODO at the lookup. The new test already covers the real path: it checks whether the method exists, and once it does, it asserts that a 2^24-window frame is rejected at 23 and decodes at 24. The settings class and the zstd API are new in 2.0.0 and `@ApiMayChange`, so there is no binary compatibility impact. `ZstdDecompressor` is `@InternalApi`. ### Tests - JDK 17: `sbt "stream-tests/testOnly org.apache.pekko.stream.io.compression.ZstdSpec org.apache.pekko.stream.io.compression.CompressionWithoutZstdSpec"` with zstd-jni 1.5.7-24: 42 passed. The new window test exercised the `UnsupportedOperationException` branch. - `sbt stream/scalafmt stream-tests/Test/scalafmt` - Not run locally: Scala 3 compile and MiMa, left to CI. ### References Refs #3674 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
