FrankYang0529 commented on PR #76279: URL: https://github.com/apache/airflow/pull/76279#issuecomment-6624095494
Thanks, I agree that stability matters here. I checked three things. - Each runner runs one job at a time. GitHub's docs say that [each standard GitHub-hosted runner is a new virtual machine](https://docs.github.com/en/actions/concepts/runners/github-hosted-runners#overview-of-github-hosted-runners), and that [each runner can run a single job at a time](https://docs.github.com/en/actions/get-started/understand-github-actions#runners). RunsOn's docs say that RunsOn starts [a fresh, ephemeral EC2 runner for every job](https://runs-on.com/docs/). In the PROD build job, no other step runs at the same time as the export step. Also, with `-T0`, zstd tries to detect the number of physical CPU cores and uses that many threads. So zstd should not start more threads than the machine has cores. - The same command already runs at scale. The CI image build has used `zstd -6 -T0` since #73997 was merged on 2026-10-01. - zstd needs less CPU time in total than the old deflate compression. On my laptop, I compressed a 3.0 GB tar of a Python virtualenv. `zstd -6` used about 17 s of CPU time, both with 1 thread and with 4 threads. `gzip -6` uses the same deflate algorithm and level as the old stash compression, and it used 55 s of CPU time on one core. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
