weiqingy opened a new issue, #1072: URL: https://github.com/apache/flink-agents/issues/1072
### Description `Skills.from_url(...)` / `Skills.fromUrl(...)` download a remote zip and extract it without bounding how many bytes get written, in either runtime. A URL skill source can therefore fill the TaskManager's temp filesystem. That is a resource exhaustion property rather than a transport or integrity one, so it is tracked apart from #1003. The download streams the response straight to a temp file with no cap, and without consulting `Content-Length`: `shutil.copyfileobj(resp, out)` at `python/flink_agents/runtime/skill/repository/_materialize.py:162`, and `Files.copy(in, tmpZip, StandardCopyOption.REPLACE_EXISTING)` at `runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:263`. A source that streams without ending, or simply serves a very large body, runs until the disk is full. Extraction is unbounded too. `extract_zip_safely` validates every entry against zip-slip and checks nothing else, then extracts: `zf.extractall(extract_dir)` at `_materialize.py:133`, and the per-entry `Files.copy(in, target)` at `SkillMaterializer.java:142`. There is no per-entry size cap, no cumulative uncompressed cap and no entry count cap, so a small archive can expand well past its transfer size. Digest pinning does not close this. In the change proposed on #1005 the SHA-256 is computed after the download helper returns, so the file is already fully on disk before any operator expectation is checked. The Java side has the same ordering. This came out of reviewing #1005 and was raised on #1003 first: https://github.com/apache/flink-agents/issues/1003#issuecomment-5383724087 ### Expected behavior - Bound the bytes written during download, failing closed with an error that names the limit. - Bound extraction on cumulative uncompressed size, per-entry size and entry count, checked before writing past the limit rather than after. - Where the server declares a `Content-Length` above the cap, reject before streaming. That is a cheap early exit, not a substitute for counting bytes as they arrive, since the header can be absent or wrong. - Keep the existing partial-output cleanup on the failure path, so a rejected download or extraction leaves nothing behind. - Keep the limits and any configuration surface aligned across Java and Python. - Add focused Java and Python tests for a download past the cap, an archive that expands past the cap, and unchanged behavior for an archive inside the limits. ### How to reproduce 1. Serve a response at an HTTPS URL that streams without ending, or whose body is far larger than any real skill archive. 2. Configure it through `Skills.from_url(...)` or `Skills.fromUrl(...)`. 3. Observe that the download runs until the source stops or the filesystem fills. Pinning a digest does not change this, because the digest is verified after the file is written. 4. Separately, serve a small valid zip whose entries expand to many gigabytes. 5. Observe that extraction proceeds entry by entry until the filesystem fills. ### Version and environment Present on `main` as of 2026-08-29 (`0417dad3`) and in 0.3.1, in both the Java and Python implementations. The line numbers above are from `main`. On the 0.3.1 tag the Java lines are `SkillMaterializer.java:264` and `:143`; the Python lines are unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
