The GitHub Actions job "Tests" on airflow.git has failed.
Run started by GitHub user potiuk (triggered by potiuk).

Head commit for run:
d8a42cadf85df026f4e41e5e440e350380615f4a / Jarek Potiuk <[email protected]>
Improve speed of regular upgrades when dependencies change. (#37360)

When regular dependencies change we enable UPGRADE_TO_NEWER_DEPENDENCIES
flag. After some recent improvements in installing dependencies from
the branch tip where we can install airflow using `[devel-ci]` extra
from the branch tip, this opens up another option for speeding up
the installations, when some dependencies change.

So far - UPGRADE_TO_NEWER_DEPENDENCIES necessitated to reinstall all
dependencies with airflow attempting to upgrade them eagerly. This
takes ~ 28 minutes on CI now from the scratch. However, this is not
needed. We can still start from the set of dependencies pre-installed
by installation from branch tip, and only trigger
"UPGRADE_TO_NEWER_DEPENDENCIES" after that - so that the "URL"
installation cache does not get invalidated.

This should limit the time needed to build such an image to ~ 8 minutes
in CI when cache is built.

In the future we might further optimize it - as we now have all the
mechanisms necessary to do that in selective checks. For example we
could trigger UPGRADE_TO_NEWER_DEPENDENCIES only when some dependencies
are changes, and not when new dependencies are added, further limiting
the time needed to build such image to some ~ 3/4 minutes. But this
should be a separate PR and should be carefuly thought about, as
it might also be that this will lead to conflicts with the latest
constraints..

Report URL: https://github.com/apache/airflow/actions/runs/7877547311

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to