potiuk commented on issue #41106: URL: https://github.com/apache/airflow/issues/41106#issuecomment-2257435367
See and read in detail the description of constraints. Constraints are only there to provide reproducible installation of Airlfow https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#constraints-files - in EXACT set of the "golden" set of requirements. If you want to upgrade any of those and change / upgrade any of the requirements of Airflow, you should have a second step of installation - where you diverge from the original constraints (and not use the constraints any more). Airflow deliberately has wide enough set of REQUIREMENTES that should allow you to diverge from the "golden set". But applying CONSTRAINTS (which has FIXED 719 or so depedencies) when at the same time you want to diverge from the constraints makes just no sense - you will always have conflicts. Constraints are not supposed to be use any more when you diverge from the "golden set". This is explained in detail here https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#constraints-files Also you can watch my talk from last Airflow summit describing the proces: https://airflowsummit.org/sessions/2023/mastering-dependencies-the-airflow-way/ Generallly - you get airlfow installed, then you update your own dependencies and then you can even generate your own constraints out of that when you arrive at good set of. Now - how this fits MWAA image and how constraints are used there, I have very little knowledge of - this is more of a question to MWAA team, but this is a general process that should be followed: 1) Install airflow with constraints to get reproducible installation 2) Modify the installation WITHOUT constraints when you want to change the dependencies 3) If you want to get it reproducible - freeze the resulting set of dependencies as "your own constraints". Converting it into discussion in case more of the discussion is needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
