mbien opened a new pull request #3901:
URL: https://github.com/apache/netbeans/pull/3901
draft PR to test some optimizations.
idea1: use Zstandard compression for pipeline artifacts.
- faster compression (multithreaded) and extraction compared to gz
- smaller artifact footprint (with `-9` setting, seems to be the sweet spot
in my experience from other deployments)
- all github nodes support it [out of the
box](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#preinstalled-software)
- cumulative performance improvement due to the nature of the pipeline
Local test using 2 threads to simulate a gh node. Tested in a ramdisk. The
actual code uses `-T0` to be forward compatible.
```bash
# compression
time tar -czf /tmp/build.tar.gz --exclude ".git" .
real 1m12,641s
user 1m18,657s
sys 0m3,167s
# uses two threads to simmulate CI node
time tar -I 'zstd -9 -T2' -cf /tmp/build.tar.zst --exclude ".git" .
real 0m22,993s
user 0m50,221s
sys 0m3,018s
# archive sizes
ls -l /tmp/build*
-rw-r--r-- 1 mbien mbien 1575854860 31. Mär 01:21 /tmp/build.tar.gz
-rw-r--r-- 1 mbien mbien 1431884800 31. Mär 01:25 /tmp/build.tar.zst
# extraction
time tar -xzf /tmp/build.tar.gz
real 0m14,681s
user 0m15,051s
sys 0m3,438s
time tar --zstd -xf /tmp/build.tar.zst
real 0m3,747s
user 0m2,399s
sys 0m3,006s
```
outside of gh actions I would have combined compression + upload in one step
(same for download + extraction). But this isn't possible while using the
default upload action.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
