potiuk commented on issue #13149: URL: https://github.com/apache/airflow/issues/13149#issuecomment-750886435
Just one comment here @maxcountryman if you are following that route. There are certain use cases that need to be solved if we would support poetry: 1) Installing from PyPI or wheel using the 'constraints' - one reason why we have constraints in our repo is that we can do the "recommended" installation method for our users. See https://github.com/apache/airflow/blob/master/docs/apache-airflow/installation.rst#getting-airflow User can run `pip install apache-airflow[EXTRAS]==1.10.14 --constratint https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt` And get a fully reproducible installation of airflow, no matter what transitional dependencies have been released since we released airlfow. The poetry.lock file is not available via PyPI and I wonder if we can direct users to install airflow from PyPI in similar way using poetry.lock ? Note that having to downloading such file before installation is not a good solution. It should be one simple command. 2) We have constraint files in orphaned branches in our repo where we can update them independently from relating Airflow. That allows us to fix and correct any kind of constraint issues we have without having to update the source code: * Master: https://github.com/apache/airflow/tree/constraints-master * 2.0: https://github.com/apache/airflow/tree/constraints-2-0 * 1.10: https://github.com/apache/airflow/tree/constraints-1-10 This allows for great flexibility much better than when constraints are committed together with the source code. This also helps us to be compliant with the release policy of Apache Software Foundation: http://www.apache.org/legal/release-policy.html because technically we do not need to modify source code/tags in github and make a formal release when we update constraints in those orphaned branches. I wonder how Poetry would cope with it and whether it will mean that we have to add a lot of custom processing to keep similar processing? By quick look at the poetry it requires poetrly.lock to be committed together with the source code and there is no tooling to help us with the "orphaned-branch" case. 3) We keep separate constraints for each python major/minor combination we support (2.7, 3.5, 3.6. 3.7, 3.8 for 1.10 and 3.6, 3.7, 3.8, and working on 3.9 in 2.0). There are subtle (but significant) differences between those, I wonder if single poetry.lock can handle this or whether we would have to have separate poetry-3.6.lock, poetry-3.7.lock etc. ). Is there any support in poetry to support that? Or again we would have to add our own tooling around that? Those are the three major challenges I see if we would like to go poetry route at some point in time - I wonder @maxcountryman (or anyone else with poetry experience) if you have some experience that could help us to see if those cases can be handled with it? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
