potiuk commented on code in PR #31882: URL: https://github.com/apache/airflow/pull/31882#discussion_r1228825582
########## docs/apache-airflow/installation/installing-from-pypi.rst: ########## @@ -79,53 +87,64 @@ where: - ``AIRFLOW_VERSION`` - Airflow version (e.g. :subst-code:`|version|`) or ``main``, ``2-0``, for latest development version - ``PYTHON_VERSION`` Python version e.g. ``3.8``, ``3.9`` -There is also a ``constraints-no-providers`` constraint file, which contains just constraints required to -install Airflow core. This allows to install and upgrade airflow separately and independently from providers. +However, that should not prevent you from being able to install providers and dependencies that were upgraded +after the release to keep your installation up-to-dated, even if you do not upgrade airflow core version. +Installing such dependencies should be done without constraints as a separate pip command. -You can create the URL to the file substituting the variables in the template below. +You should make sure to also add the ``apache-airflow`` package to the list of packages to install and pin +it to the version that you have, otherwise you might end up with a different version of Airflow than you +expect as ``pip`` can upgrade/downgrade it automatically when performing dependency resolution. -.. code-block:: +.. code-block:: bash - https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-no-providers-${PYTHON_VERSION}.txt + pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.8.txt" + pip install "apache-airflow[celery]==|version|" apache-airflow-providers-google==10.1.1 -You can also use "latest" as version when you install "latest" stable version of Airflow. The "latest" -constraints always points to the "latest" released Airflow version constraints: +You can also downgrade providers or other dependencies this way if you need to (for example if you do not +want to upgrade to the latest version of a provider). -.. code-block:: +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.8.txt" + pip install "apache-airflow[celery]==|version|" dbt-core==0.20.0 + + +Not always such upgrade is possible - you might have conflicting dependencies, but by not using constraints, +you give ``pip`` a chance to resolve the conflicts for you, while keeping within the limits of dependencies +that Apache Airflow and other providers require. The combination of those dependencies might not be as well +tested as the "golden" set of dependencies, but it should work in most cases. You can also always run +the ``pip check`` command to test if the set of your Python packages is consistent and not conflicting. + + +Using your own constraints +========================== + +When you decide to install your own dependencies, or want to upgrade or downgrade providers, you might want +to continue being able to run reproducible installation of Airflow and those dependencies. In order to do +that you can produce your own constraints file and use it to install Airflow instead of the one provided +by the community. + +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.8.txt" + pip install "apache-airflow[celery]==|version|" dbt-core==0.20.0 + pip freeze > my-constraints.txt + + +Then you can use it to run reproducible installation of your environment in a single operation via local file: + +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "my-constraints.txt" + + +The constraint file might also be hosted via a webserver of your choice and made available for remote use: + + +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "https://my-company.org/my-constraints.txt"; - https://raw.githubusercontent.com/apache/airflow/constraints-latest/constraints-3.8.txt - - -Fixing Constraint files at release time Review Comment: Yes. That was a bit too much, but I simplified it, removed middles section (which was a bit bullshit), and referred to "use your own constraints" in case the users want to keep updated between the released. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
