potiuk commented on PR #629: URL: https://github.com/apache/airflow-site/pull/629#issuecomment-1179396727
> Yeah, I think this could be possible. With homebrew/apt/yum and venv we can probably create a fairly portable build environment. But I still find a lot of benefit in a containerized process. It ensures consistency between the dev's local environment and the one inside CI. > Right now the build is failing on this PR and I think it's because I updated python to python3 in a few spots in site.sh. I did this because one of the systems I test on has both python 2 and 3 installed, and the symlink for python was pointing to 2. I feel it's better to be explicit about python versions, generally. Of course, now I am solving a problem on my local machine that doesn't exist in CI, and along the way I am now suspecting I might have broken CI! There are some problems with python3 aliases and expecting them is not universal even in https://peps.python.org/pep-0394/ there is a freedom to decide which alias is pointing to which version and I've seen cases where python3 was missing as well (this is left to a discretion of system administrators and distribution). Hoever "future-looking" approach is that Python2 will all but disappear and python alias will point to python3. And then there will also be a movement to remove python3 think even if it is there. So in all airlfow code we tend to use: ``` #!/usr/bin/env python ``` Though occasionally you will find ``` #!/usr/bin/env python3 ``` BTW. In most distros you can install`python-is-python3` package to get this behaviour forced: ``` sudo apt install python-is-python3 ``` > This wouldn't be a problem if those steps in site.sh ran inside the container. But the current process spans the container + the local environment. So we're somewhat missing the benefit. > On one of the other sites I manage, we support local environments as you suggest. It's easy to work with. However, one consequence is that certain components of node behave differently on different architectures, leading to YAML files that contain the same information but constantly switch order, etc., creating lots of git noise. All of the above I wholeahartedly agree with and I have exactly the same experience. I am (as you might notice from Airflow CI and Breeze) huge Docker fan for CI + dev env for precisely the reasons you describe. So if `buster` migration allows to get "simple" Docker env, this is certainly the best approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
