Jarek, wow. Seems to be a great improvement for Breeze and pytests. Great news.
Best regards On Mon, Jan 13, 2020 at 10:57 AM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > TL;DR; I have a Green and ready for review PR that should vastly improve > the stability of CI and make our development environment much more > enjoyable: https://github.com/apache/airflow/pull/7091 > > From the first experiences it is much more stable on Travis and we also > have a working version with Github Actions so we should be able to move to > GA shortly. > > It brings huge improvements to CI environment as well to local development > one. It will be much easier and more convenient to reproduce failing CI > tests. > > This is all documented in TESTING.rst and BREEZE.rst but here some summary > of what I came up with. > > *Using pytest markers to separate Integration/Kubernetes tests:* > > 1) Appropriate tests are marked now with > *pytest.mark.integration("<integration>")* marker (*cassandra kerberos > mongo openldap rabbitmq redis*) - in case you do not have an integration > enabled they will be skipped and the message will clearly say what > integration is needed. You can also run only the tests for those > integrations only with *pytest --integration cassandra* for example of > *pytest > --integration all* > > 2) There are some tests that require/work with certain backends only > (*sqlite > mysql postgres*). They are marked with *pytest.markers.backend("mysql")* or > even *pytest.mark.backend("mysql", "postgres")* - if they are supposed to > work with more than one backend. There are ~30 such tests. I corrected all > custom skips/skipifs/skipunless to those new markers. Same as with > integrations you can run all backend tests with *pytest --backend mysql* > > 3) There are Kubernetes tests that are marked with > *pytest.mark.runtime("kubernetes") > *. I chose a different marker (runtime) for those because they are > completely independent and require more fundamental environment change - > starting and deploying Kubernetes cluster. There are ~30 such tests. Again > you can run only those tests with *pytest --runtime kubernetes.* > > 4) Breeze is now by default not running any additional > containers/integrations - only base airflow. There are *~ 4400 *tests that > should run in that environment not requiring any integrations (i.e. other > containers) or runtime (i.e. kubernetes). So vast majority of the tests > will run with this default setup. This makes the whole "Breeze" experience > a lot more enjoyable as the default setting requires far less resources on > your local machine. > > 5) You can also start Breeze with *--integration* flag (for example > *./breeze > --integration cassandra*) so that you have the integration to run > appropriate integration tests (for example cassandra tests). > > 6) You can also start Breeze with "*--start-kind-cluster*" . This will > start "Kubernetes in Docker" cluster. And with two scripts you can deploy > Airflow to this cluster and run Kubernetes tests there. > > 7) Last but not least - we have more jobs in Travis (soon moving to Github > Actions). These jobs should be far more stable, because either they do not > require integrations to be started (so much less memory needed) or they run > only integrations/kubernetes tests (they run 30 tests each so the memory > requirement is also much smaller). > > We see much better now which tests are skipped and why and it will allow us > to further cleanup our test base now. > > You can see a successful build here: > https://travis-ci.org/apache/airflow/builds/636205535 > > Looking forward to prompt review and merging it quickly. > > J. > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > -- Michał Słowikowski Polidea <https://www.polidea.com/> | Test Engineer E: michal.slowikow...@polidea.com Unique Tech Check out our projects! <https://www.polidea.com/our-work>
