agjendem commented on PR #1815: URL: https://github.com/apache/maven-mvnd/pull/1815#issuecomment-6165183901
A bit more detail on why the check fails for a byte-identical file. In short, it looks like a clash between the `InputStream.read` specification and an optimisation in zlib-ng. `contentsEquals` reads both files in 8 KiB blocks and compares the two buffers. In the last block, only the start of each buffer holds new data, and the rest still holds the previous block. The old code compared the whole buffer, so it also compared those leftover bytes. That is safe as long as `read()` leaves the buffer past the returned bytes alone, which is what the `InputStream.read` specification promises. zlib-ng doesn't seem to make that promise. To go faster, it copies data in fixed-size chunks, and a chunk can reach a few bytes past the end of the data, as long as it stays inside the output space ([`inffast_tpl.h`](https://github.com/zlib-ng/zlib-ng/blob/2.3.3/inffast_tpl.h#L270-L273)). The JDK hands it the caller's buffer as that output space, so when the JDK reads the jar entry through zlib-ng, a few bytes just after the returned data change. In the middle of the file the next read writes over them, but after the last block they stay, and the check fails even though the data is identical. For example, the Linux x86_64 `libmvndnative.so` is 8384 bytes, so the last block has 192 bytes. With Corretto on Fedora 44, the read that returns those 192 bytes also changes 7 of the bytes right after them. Comparing only the bytes that were read does not depend on that promise, so it works with any zlib. Loading `CLibrary` from the `mvnd-native` jar, built from `mvnd-1.x` and from this branch: | Setup | zlib-ng | `mvnd-1.x` | This PR | |---|---|---|---| | Fedora 44, Corretto 25.0.4 | yes (system zlib) | fails with `Content differs` | loads | | Fedora 44, Fedora OpenJDK 25.0.4 | no | loads | loads | | macOS (arm64), Corretto 25.0.4 | no | loads | loads | In all three setups, reading the Linux x86_64 entry gives the same 192-byte last block, but only with zlib-ng do bytes past it change, and only that setup fails. It is the same `CLibrary` initialization failure as in #1262 and #1438. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
