insideo commented on pull request #2513: URL: https://github.com/apache/hadoop/pull/2513#issuecomment-744741998
@ericbadger I suspect the performance delta is due to the latest mksquashfs code being multi-threaded during encodes -- the larger images seem to have a higher deltas in your example. We could implement multi-threaded conversion in the Java code as well, but since the engine is designed to be mostly streaming, it would be a pretty big code change. Also, this would make reproducible builds considerably more difficult to ensure. What we could do is process multiple layers in parallel - this would likely close the gap since most real-world images have several layers which would need conversion, and since each individual layer would still be processed serially, reproducibility would be maintained. Thoughts? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
