[
https://issues.apache.org/jira/browse/AIRFLOW-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796258#comment-16796258
]
Jarek Potiuk commented on AIRFLOW-2556:
---------------------------------------
One comment. While not improving the individual test times, there might be some
improvements in test setup time with
[https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-10+Multi-layered+and+multi-stage+official+Airflow+image#AIP-10Multi-layeredandmulti-stageofficialAirflowimage-Buildtimings]
I analysed build overhead (setup time) in various cases in my POC.
I did not yet check definitely how much the setup takes in the current build.
But I believe it is around 5 minutes from observation of running buids - to
pull docker images + install dependencies from .wheel cache currently - it's
hard to say for sure as this is not a separately timed script/step. I might
measure it precisely tonight and let you know.
In the multi-stage builds setup time is usually between 2:30 and 6 minutes to
pull images and rebuild changed layers. For vast majority of cases where only
sources change it takes less than 3 minutes. Then 6 minutes it takes only when
dependencies are changed and 8 + minutes if APT dependencies change - all of
which should be quite rare case).
I yet have to time the whole build and see overall impact after i fix all
failing tests in the new environment. But that might be another small
improvement as it will impact all the jobs.
> Reduce time spent on unit tests
> -------------------------------
>
> Key: AIRFLOW-2556
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2556
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: Bolke de Bruin
> Priority: Major
>
> Unit tests are taking up way too much time. This costs time and actually also
> Money from the Apache Foundation. We need to reduce this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
