pankajkoti commented on code in PR #32948: URL: https://github.com/apache/airflow/pull/32948#discussion_r1278557402
########## TESTING.rst: ########## @@ -1442,6 +1442,88 @@ You can also run complete k8s tests with This will create cluster, build images, deploy airflow run tests and finally delete clusters as single command. It is the way it is run in our CI, you can also run such complete tests in parallel. +Manually testing release candidate packages +=========================================== + +Breeze can be used to test new release candidates of packages - both Airflow and providers. You can easily +turn the CI image of Breeze to install and start airflow for both Airflow and provider packages - both, +packages that are built from sources and packages that are downloaded from PyPI when they are released +there as release candidates. + +The way to test it is rather straightforward: + +1) Make sure that the packages - both ``airflow`` and ``providers`` are placed in the ``dist`` folder + of your Airflow source tree. You can either build them there or download from PyPI (see the next chapter) + +2) You can run ```breeze shell`` or ``breeze start-airflow`` commands with adding the following flags - + ``--mount-sources remove`` and ``--use-packages-from-dist``. The first one removes the ``airflow`` + source tree from the container when starting it, the second one installs ``airflow`` and ``provider`` + packages from the ``dist`` folder when entering breeze. + +Testing pre release packages Review Comment: ```suggestion Testing pre-release packages ``` ########## TESTING.rst: ########## @@ -1442,6 +1442,88 @@ You can also run complete k8s tests with This will create cluster, build images, deploy airflow run tests and finally delete clusters as single command. It is the way it is run in our CI, you can also run such complete tests in parallel. +Manually testing release candidate packages +=========================================== + +Breeze can be used to test new release candidates of packages - both Airflow and providers. You can easily +turn the CI image of Breeze to install and start airflow for both Airflow and provider packages - both, +packages that are built from sources and packages that are downloaded from PyPI when they are released +there as release candidates. + +The way to test it is rather straightforward: + +1) Make sure that the packages - both ``airflow`` and ``providers`` are placed in the ``dist`` folder + of your Airflow source tree. You can either build them there or download from PyPI (see the next chapter) + +2) You can run ```breeze shell`` or ``breeze start-airflow`` commands with adding the following flags - + ``--mount-sources remove`` and ``--use-packages-from-dist``. The first one removes the ``airflow`` + source tree from the container when starting it, the second one installs ``airflow`` and ``provider`` + packages from the ``dist`` folder when entering breeze. + +Testing pre release packages Review Comment: hope this does not hamper the links for https://github.com/apache/airflow/blob/main/TESTING.rst#testing-pre-release-packages ########## TESTING.rst: ########## @@ -1442,6 +1442,88 @@ You can also run complete k8s tests with This will create cluster, build images, deploy airflow run tests and finally delete clusters as single command. It is the way it is run in our CI, you can also run such complete tests in parallel. +Manually testing release candidate packages +=========================================== + +Breeze can be used to test new release candidates of packages - both Airflow and providers. You can easily +turn the CI image of Breeze to install and start airflow for both Airflow and provider packages - both, +packages that are built from sources and packages that are downloaded from PyPI when they are released +there as release candidates. + +The way to test it is rather straightforward: + +1) Make sure that the packages - both ``airflow`` and ``providers`` are placed in the ``dist`` folder + of your Airflow source tree. You can either build them there or download from PyPI (see the next chapter) + +2) You can run ```breeze shell`` or ``breeze start-airflow`` commands with adding the following flags - + ``--mount-sources remove`` and ``--use-packages-from-dist``. The first one removes the ``airflow`` + source tree from the container when starting it, the second one installs ``airflow`` and ``provider`` + packages from the ``dist`` folder when entering breeze. + +Testing pre release packages +---------------------------- + +There are two ways how you can get airflow packages in ``dist`` folder - by building them from sources or +downloading them from PyPI. + +.. note :: + + Make sure you run ``rm dist/*`` before you start building packages or downloading them from PyPI because + the packages built there already are not removed manually. + +In order to build apache-airflow from sources, you need to run the following command: + +.. code-block:: bash + + breeze release-managment prepare-airflow-package + +In order to build providers from sources, you need to run the following command: + +.. code-block:: bash + + breeze release-managment prepare-provider-packages <PROVIDER_1> <PROVIDER_2> ... <PROVIDER_N> + +The packages are built in ``dist`` folder and the command will summarise what packages are available in the +``dist`` folder after it finishes. + +If you want to download the packages from PyPI, you need to run the following command: + +.. code-block:: bash + + pip download apache-airflow-providers-<PROVIDER_NAME>==X.Y.Zrc1 --dest dist --no-deps + +You can use it for both release and pre-release packages. + + +Few examples below will explain how you can test pre-release (also release) packages. + +This one will download ``airflow`` and ``celery`` and ``kubernetes`` provider packages from PyPI and +eventually start Airflow using the packages downloaded with the Celery Executor. It will also +load example dags and default connections: + +.. code:: bash + + rm dist/* + pip download apache-airflow==2.7.0rc1 --dest dist --no-deps + pip download apache-airflow-providers-cncf-kubernetes==7.4.0rc1 --dest dist --no-deps + pip download apache-airflow-providers-cncf-kubernetes==3.3.0rc1 --dest dist --no-deps + breeze start-airflow --mount-sources remove --use-packages-from-dist --executor CeleryExecutor --load-default-connections --load-example-dags + + +This one will download ``airflow`` and ``celery`` and ``kubernetes`` provider packages from PyPI but build Review Comment: Maybe this one will not download ``airflow`` from PyPI? ########## dev/README_RELEASE_AIRFLOW.md: ########## @@ -416,9 +421,9 @@ EOF Note, For RC2/3 you may refer to shorten vote period as agreed in mailing list [thread](https://lists.apache.org/thread/cv194w1fqqykrhswhmm54zy9gnnv6kgm). -# Verify the release candidate by PMCs +# Verify the release candidate by PMC members -The PMCs should verify the releases in order to make sure the release is following the +The PMC members should verify the releases in order to make sure the release is following the Review Comment: ```suggestion PMC members should verify the releases in order to make sure the release is following the ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
