allthingssecurity opened a new pull request, #28597: URL: https://github.com/apache/camel/pull/28597
# Description [CAMEL-26089](https://issues.apache.org/jira/browse/CAMEL-26089) The Zip File and Tar File data formats limit how much data an unmarshal may decompress: `maxDecompressedSize` (default 1073741824 bytes = 1 GiB, `-1` for no limit, an `IOException` when it is exceeded). The two data formats of camel-zip-deflater, `gzipDeflater` and `zipDeflater`, copy the whole decompressed stream with `IOHelper.copy(InputStream, OutputStream)` and have no such option. This PR gives them the same option, for parity with zipFile and tarFile. Changes: - `GzipDeflaterDataFormat` and `ZipDeflaterDataFormat` (camel-zip-deflater) get `maxDecompressedSize`: a `long`, default 1073741824, and `-1` (any value of 0 or less) for no limit, as in `ZipFileDataFormat`. `unmarshal` copies through the bounded `IOHelper.copy(InputStream, OutputStream, int, boolean, long)` overload that `ZipFileDataFormat.unmarshal` uses, so an unmarshal over the limit fails with the same `IOException` ("The InputStream entry being copied exceeds the maximum allowed size"). That overload counts in a `long` since CAMEL-25023, so a limit of 2 GiB or more is enforced. The copy is in a small package-private method, `copyDecompressed`, so the test can feed it a generated stream. - The model classes `GzipDeflaterDataFormat` and `ZipDeflaterDataFormat` (camel-core-model) get the attribute (`java.lang.Long`, label `advanced`, default `1073741824`) and `Builder.maxDecompressedSize(String/long)`; the two reifiers pass it on. The model `GzipDeflaterDataFormat` had no attribute before, so it now has `@XmlAccessorType(XmlAccessType.FIELD)` like `ZipDeflaterDataFormat`, and its `Builder.end()` uses a new private constructor. - Regenerated: the configurers and data format JSON of camel-zip-deflater, the model JSON, `ModelParser`, the XML, YAML and Java DSL model writers, the YAML DSL deserializers and the three YAML DSL schemas, and the catalog (data format and model JSON, `camel-xml-io.xsd`, `camel-spring.xsd`, `camelYamlDsl-model.json`). - Marshal is not changed. - Upgrade guide 4.24: `camel-zip-deflater - gzipDeflater and zipDeflater limit the decompressed size`. With the same default as zipFile, a message that decompresses to more than 1 GiB now fails where it worked before; the entry says to set a higher limit or `-1`, and that the limit counts the whole decompressed message (for a `.tar.gz` unmarshalled with `gzipDeflater` and then split into its entries, the whole tar archive, where the `tarFile` limit applies to each entry). Default: I used the zipFile/tarFile default (1 GiB) rather than "no limit", so that the option has the same name, default and meaning in all four data formats. The cost is the behaviour change above, for messages of more than 1 GiB after decompression. The users most likely to see it unmarshal large `.tar.gz` files with `gzipDeflater` and split the tar afterwards (the use case of CAMEL-16953): here the limit counts the whole archive, while zipFile and tarFile count each entry. If you prefer to keep these two data formats unlimited by default, it is a one-line change in each class and in the model metadata, and the guide entry goes away. Cost: unmarshal now copies through the buffer loop of `IOHelper.copy` (as zipFile does) instead of `InputStream.transferTo`, with one `long` comparison per buffer. As with zipFile, the check runs after each write, so up to one buffer more than the limit can reach the output before the exception. Tests: - New `DeflaterMaxDecompressedSizeTest`, 7 tests, JUnit assertions as in the module: - routes with `dataFormat().gzipDeflater().maxDecompressedSize(...).end()` and the zipDeflater equivalent, so through the model and the reifier: a limit of 10 bytes, and one byte less than the message, fail with the `IOException` and the message above, and nothing reaches the mock; a limit equal to the message length, `-1`, and 4 GiB + 10 bytes (an `int` cast would make it 10) return the original bytes; - on the data formats directly, with a generated stream of zeros that holds no data and an output that only counts bytes: the default is 1 GiB (exactly 1 GiB passes, one byte more fails), a limit of 2 GiB + 1 byte passes exactly that many bytes and fails one byte more, and `-1` passes more than 1 GiB. - On main the new test does not compile (the option does not exist). Mutations, run on this commit in a separate worktree: - gzipDeflater copying with the unbounded `IOHelper.copy(InputStream, OutputStream)`: `gzipUnmarshalOverTheLimitFails` ("Expected org.apache.camel.CamelExecutionException to be thrown, but nothing was thrown"), `defaultLimitIsOneGibibyte` and `limitOverTwoGibibytesIsEnforced` ("Expected java.io.IOException to be thrown, but nothing was thrown") fail; - zipDeflater alone passing `(int) maxDecompressedSize` (gzipDeflater unchanged): `zipUnmarshalAtTheLimitOrWithoutLimitWorks` (the 4 GiB + 10 bytes limit becomes 10, `CamelExecutionException`) and `limitOverTwoGibibytesIsEnforced` ("Expected java.io.IOException to be thrown, but nothing was thrown") fail; - the other tests pass under these mutations, as they should: they check values that the mutations keep (`gzipUnmarshalAtTheLimitOrWithoutLimitWorks` and `noLimitCopiesMoreThanTheDefault` with no or a large limit, `zipUnmarshalOverTheLimitFails` with limits that fit in an `int`). - Two more mutations, each with the whole reactor rebuilt: both data formats copying with the unbounded `IOHelper.copy(InputStream, OutputStream)` (the copy of main, with the option still there): 4 of 7 fail (`gzipUnmarshalOverTheLimitFails`, `zipUnmarshalOverTheLimitFails`, `defaultLimitIsOneGibibyte`, `limitOverTwoGibibytesIsEnforced`); the two reifiers not passing the option on: 2 of 7 fail (`gzipUnmarshalOverTheLimitFails`, `zipUnmarshalOverTheLimitFails`). - camel-zip-deflater with the change: 23 tests, 0 failures. - I built offline with `-Djackson3-version=3.2.3` and `-Dasm-version=9.11` (not in my local repository at the versions of the pom). What was not run: only the tests of camel-zip-deflater, with JDK 21. No XML or YAML route using the new attribute was run. My local generator plugins are older than main and changed unrelated lines when I regenerated; I kept only the lines of this option. The YAML DSL deserializers and schemas got the option without its description, default and display name, because the YAML generator reads them from a camel-catalog jar that does not have the option yet; I filled them in from the regenerated model JSON, in the form the generator writes for the zipFile option. `camel-spring.xsd` was not regenerated (camel-spring-xml does not build offline here); it has the same two attributes as the regenerated `camel-xml-io.xsd`, which the build generates from the same model. CI's uncommitted-changes check will show if any of these differ. A rebuild of the core, XML/YAML/Java IO and YAML DSL modules at the `test` phase left the model JSON, `ModelParser`, the model writers, the configurers and the data format JSON of this option unchanged; only the YAML DSL files lost the description and default again, for the reason above. Not changed: other places that decompress (the lzf data format, and the gzip handling of HTTP request bodies) are out of scope; I can open a JIRA for lzf if you want the same option there. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26089](https://issues.apache.org/jira/browse/CAMEL-26089). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built the core modules, the YAML DSL modules and camel-zip-deflater, including the formatter and import-sort plugins, and committed the regenerated files as described above. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
