baohe-zhang commented on pull request #28412:
URL: https://github.com/apache/spark/pull/28412#issuecomment-655165640
@HeartSaVioR @tgravescs
These are the measurement of memory usage on zstd compressed log files:
* 200 jobs, 400 tasks for each job: 8.6 MB file size, 50.5 MB memory usage.
* 100 jobs, 400 tasks for each job: 4.5 MB file size, 31.4 MB memory usage.
* 50 jobs, 400 tasks for each job: 2.4 MB file size, 15. 7 MB memory usage.
* 10 jobs: 400 tasks for each job: 816 KB file size, 4.7 MB memory usage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
