xmg333 opened a new issue, #18459: URL: https://github.com/apache/dolphinscheduler/issues/18459
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened When a user downloads a large task log, the apiserver can run out of memory. The whole RPC response body is read into memory at once. In our heap dump we saw a `TransporterDecoder` whose `body` was a `byte[]` of ~226MB — it was a `queryTaskResult` response. On 3.3.2 the decoder does `new byte[bodyLength]` without any size check, so any large response gets allocated whole. Also, once a `maxFrameSize` guard is in place, a task log larger than the frame limit can no longer be downloaded — the log is returned as one big `byte[]` in a single RPC, so the request is rejected. ### What you expected to happen Large task logs should still be downloadable (streamed in chunks, not one big RPC), and the server shouldn't OOM on a large response. ### How to reproduce 1. Run a task that produces a large log file. 2. Click "Download full log" or view the task result. 3. On 3.3.2 the apiserver OOMs; with the frame guard, the large-log download fails. ### Anything else The OOM side is handled by the `maxFrameSize` check in `TransporterDecoder`. This issue also tracks making large-log download work under that limit (stream the log in chunks). ### Version 3.3.2 ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
