potiuk commented on code in PR #31882: URL: https://github.com/apache/airflow/pull/31882#discussion_r1229111800
########## docs/apache-airflow/installation/installing-from-pypi.rst: ########## @@ -79,50 +87,86 @@ where: - ``AIRFLOW_VERSION`` - Airflow version (e.g. :subst-code:`|version|`) or ``main``, ``2-0``, for latest development version - ``PYTHON_VERSION`` Python version e.g. ``3.8``, ``3.9`` -There is also a ``constraints-no-providers`` constraint file, which contains just constraints required to -install Airflow core. This allows to install and upgrade airflow separately and independently from providers. +**However, that should not prevent you from being able to upgrade or downgrade providers and dependencies to other versions** -You can create the URL to the file substituting the variables in the template below. +You can, for example, install new versions of providers and dependencies after the release to use the latest +version and up-to-date with latest security fixes - even if you do not want upgrade airflow core version. +Or you can downgrade some dependencies or providers if you want to keep previous versions for compatibility +reasons. Installing such dependencies should be done without constraints as a separate pip command. -.. code-block:: +When you do such an upgrade, you should make sure to also add the ``apache-airflow`` package to the list of +packages to install and pin it to the version that you have, otherwise you might end up with a +different version of Airflow than you expect as ``pip`` can upgrade/downgrade it automatically when +performing dependency resolution. - https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-no-providers-${PYTHON_VERSION}.txt +.. code-block:: bash -You can also use "latest" as version when you install "latest" stable version of Airflow. The "latest" -constraints always points to the "latest" released Airflow version constraints: + pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.8.txt" + pip install "apache-airflow[celery]==|version|" apache-airflow-providers-google==10.1.1 -.. code-block:: +You can also downgrade or upgrade other dependencies this way - even if they are not compatible with +those dependencies that are stored in the original constraints file: + +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.8.txt" + pip install "apache-airflow[celery]==|version|" dbt-core==0.20.0 - https://raw.githubusercontent.com/apache/airflow/constraints-latest/constraints-3.8.txt + +Such an upgrade is not always possible - you might have conflicting dependencies. However, by not using constraints, +you give ``pip`` a chance to resolve the conflicts for you, while keeping within the limits of dependencies +that Apache Airflow and other providers require. The combination of those dependencies might not be as well +tested as the "golden" set of dependencies, but it should work in most cases. You can also always run +the ``pip check`` command to test if the set of your Python packages is consistent and not conflicting. + + +Using your own constraints +========================== + +When you decide to install your own dependencies, or want to upgrade or downgrade providers, you might want +to continue being able to keep reproducible installation of Airflow and those dependencies via single command +line . In order to do that, you can produce your own constraints file and use it +to install Airflow instead of the one provided by the community. + +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.8.txt" + pip install "apache-airflow[celery]==|version|" dbt-core==0.20.0 + pip freeze > my-constraints.txt + + +Then you can use it to create reproducible installations of your environment in a single operation via +a local constraints file: + +.. code-block:: bash + + pip install "apache-airflow[celery]==|version|" --constraint "my-constraints.txt" + + +The constraint file might also be hosted via a webserver of your choice and made available for remote use: Review Comment: I will just make a general statement for it - no example is needed indeed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
