Jarek, wow. Seems to be a great improvement for Breeze and pytests. Great

Best regards

On Mon, Jan 13, 2020 at 10:57 AM Jarek Potiuk <jarek.pot...@polidea.com>

> TL;DR; I have a Green and ready for review PR that should vastly improve
> the stability of CI and make our development environment much more
> enjoyable: https://github.com/apache/airflow/pull/7091
> From the first experiences it is much more stable on Travis and we also
> have a working version with Github Actions so we should be able to move to
> GA shortly.
> It brings huge improvements to CI environment as well to local development
> one. It will be much easier and more convenient to reproduce failing CI
> tests.
> This is all documented in TESTING.rst and BREEZE.rst but here some summary
> of what I came up with.
> *Using pytest markers to separate Integration/Kubernetes tests:*
> 1) Appropriate tests are marked now with
> *pytest.mark.integration("<integration>")* marker (*cassandra kerberos
> mongo openldap rabbitmq redis*) - in case you do not have an integration
> enabled they will be skipped and the message will clearly say what
> integration is needed. You can also run only the tests for those
> integrations only with *pytest --integration cassandra* for example of
> *pytest
> --integration all*
> 2) There are some tests that require/work with certain backends only
> (*sqlite
> mysql postgres*). They are marked with *pytest.markers.backend("mysql")* or
> even *pytest.mark.backend("mysql", "postgres")* - if they are supposed to
> work with more than one backend.  There are ~30 such tests. I corrected all
> custom skips/skipifs/skipunless to those new markers. Same as with
> integrations you can run all backend tests with *pytest --backend mysql*
> 3) There are Kubernetes tests that are marked with
> *pytest.mark.runtime("kubernetes")
> *. I chose a different marker (runtime) for those because they are
> completely independent and require more fundamental environment change -
> starting and deploying Kubernetes cluster. There are ~30 such tests. Again
> you can run only those tests with *pytest --runtime kubernetes.*
> 4) Breeze is now by default not running any additional
> containers/integrations - only base airflow. There are *~ 4400 *tests that
> should run in that environment not requiring any integrations (i.e. other
> containers) or runtime (i.e. kubernetes). So vast majority of the tests
> will run with this default setup. This makes the whole "Breeze" experience
> a lot more enjoyable as the default setting requires far less resources on
> your local machine.
> 5) You can also start Breeze with *--integration* flag (for example
> *./breeze
> --integration cassandra*) so that you have the integration to run
> appropriate integration tests (for example cassandra tests).
> 6) You can also start Breeze with "*--start-kind-cluster*" . This will
> start "Kubernetes in Docker" cluster. And with two scripts you can deploy
> Airflow to this cluster and run Kubernetes tests there.
> 7) Last but not least - we have more jobs in Travis (soon moving to Github
> Actions). These jobs should be far more stable, because either they do not
> require integrations to be started (so much less memory needed) or they run
> only integrations/kubernetes tests (they run 30 tests each so the memory
> requirement is also much smaller).
> We see much better now which tests are skipped and why and it will allow us
> to further cleanup our test base now.
> You can see a successful build here:
> https://travis-ci.org/apache/airflow/builds/636205535
> Looking forward to prompt review and merging it quickly.
> J.
> --
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>


Michał Słowikowski
Polidea <https://www.polidea.com/> | Test Engineer

E: michal.slowikow...@polidea.com

Unique Tech
Check out our projects! <https://www.polidea.com/our-work>

Reply via email to