potiuk commented on pull request #16169: URL: https://github.com/apache/airflow/pull/16169#issuecomment-851014021
> Thanks @potiuk. Do you have any details for these "self-hosted runners"? E.g what OS are they running on and using what version of python? Yeah. Sorry I have not added the links. I update the description in a moment. It seems for all versions of python. The system is based on ubuntu - here are all the `packer` scripts that prepare the runner instance: https://github.com/apache/airflow-ci-infra/tree/main/github-runner-ami/packer What is different for those instances is that they are much bigger than runner (64GB RAM, 8VCPUS vs 6GB/2VCPUs) and what we implemented there is that most of the flesystem used is tmpfs-backed: ``` sudo mount -t tmpfs -o size=10% tmpfs /tmp sudo mount -t tmpfs -o size=66% tmpfs /var/lib/docker sudo mount -t tmpfs -o tmpfs /home/runner/actions-runner/_work ``` That makes everything much faster and has much less chance of being synchronized on whatever file acccess.. I presume that might be the biggest reason for observed differences. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
