Just skimmed through the PR and looks good. Good work Tomek. I will have do a detailed review soon (if not today, then Monday).
Regards, Kaxil On Fri, Nov 29, 2019, 14:06 Jarek Potiuk <jarek.pot...@polidea.com> wrote: > Just to add to what Tomek wrote - I love the change! > > I was testing it and it's great because we get rid of all the external > scripts to run the tests. All the "airflow-specific" code is added as > pytest extension and it also means that integration with a number of tools > (like IDE's/CI etc.) will be possible. > > One thing we think about - it will enable (in the near future) to be able > to run and debug your DAGs directly in Breeze image using your favourite > IDE with breakpoints etc. - without having to use remote debugging. > Also some integration with pre-commits will be possible. And some further > modernization of our developer tools with regard to tests will be possible. > > J. > > On Fri, Nov 29, 2019 at 2:42 PM Tomasz Urbaszek < > tomasz.urbas...@polidea.com> > wrote: > > > It seems that we are able to migrate from nose to pytest. As mentioned > > earlier in this thread it seems > > wise to merge the PR as soon as possible. It's a rather fundamental > change > > so I would target it for next > > week instead of doing it on Friday. > > > > When we switch to pytest we should be able to use all its features > > including marking tests and custom > > plugins (that already helped to remove some scripts). Next step I would > > like to take is to extract tests that > > rely on database and isolate them with fixtures so we can start > eliminating > > side effects. > > > > From developer perspective the change means mostly that you should use > > pytest instead of nose / run-tests > > script. If you run test from IDE it will mean that you need to have to > > adjust test runner. > > > > The mentioned PR: https://github.com/apache/airflow/pull/6472 > > > > Bests, > > Tomek > > > > On Wed, Nov 27, 2019 at 5:11 PM Tomasz Urbaszek < > > tomasz.urbas...@polidea.com> > > wrote: > > > > > Hi all, > > > > > > Here is a PR that introduces pytest. Your opinions are highly > > appreciated! > > > ;) > > > https://github.com/apache/airflow/pull/6472 > > > > > > Travis's red due to one test (test_on_kill) but it can be fixed as seen > > in > > > previous commits. > > > > > > Bests, > > > Tomek > > > > > > On Thu, Oct 31, 2019 at 6:11 PM Chao-Han Tsai <milton0...@gmail.com> > > > wrote: > > > > > >> +1 > > >> > > >> On Wed, Oct 30, 2019 at 5:37 PM Felix Uellendall > <felue...@pm.me.invalid > > > > > >> wrote: > > >> > > >> > +1 of course 👍 > > >> > > > >> > Felix > > >> > > > >> > Sent from ProtonMail Mobile > > >> > > > >> > On Wed, Oct 30, 2019 at 17:31, Kaxil Naik <kaxiln...@gmail.com> > > wrote: > > >> > > > >> > > And yes we should merge it as soon as we have all green > > >> > > > > >> > > On Wed, Oct 30, 2019 at 4:28 PM Kaxil Naik <kaxiln...@gmail.com> > > >> wrote: > > >> > > > > >> > >> +1 pytest is way better > > >> > >> > > >> > >> On Wed, Oct 30, 2019 at 3:59 PM Driesprong, Fokko > > >> <fo...@driesprong.frl > > >> > > > > >> > >> wrote: > > >> > >> > > >> > >>> +1 Can't wait! > > >> > >>> > > >> > >>> Op wo 30 okt. 2019 om 16:57 schreef Ash Berlin-Taylor < > > >> a...@apache.org > > >> > >: > > >> > >>> > > >> > >>> > I'm not sure this even warrants an AIP. What do people think? > > Once > > >> > the > > >> > >>> > tests are passing (and making them pass is a good thing as it > > >> means > > >> > they > > >> > >>> > are less reliant on side-effects or ordering from previous > > tests) > > >> > this > > >> > >>> > isn't that big a change, right? > > >> > >>> > > > >> > >>> > (+1 though. Love py-test) > > >> > >>> > > > >> > >>> > -a > > >> > >>> > > > >> > >>> > > On 30 Oct 2019, at 14:49, Jarek Potiuk < > > >> jarek.pot...@polidea.com> > > >> > >>> wrote: > > >> > >>> > > > > >> > >>> > > I think it's the right time to vote - I think the benefits > are > > >> > obvious > > >> > >>> > and > > >> > >>> > > seeing the number of problems you already fixed (I followed > > the > > >> > draft > > >> > >>> PR) > > >> > >>> > > it is quite doable to finish that in a short time. > > >> > >>> > > > > >> > >>> > > Just in case (+1 binding) from my side. > > >> > >>> > > > > >> > >>> > > J. > > >> > >>> > > > > >> > >>> > > On Wed, Oct 30, 2019 at 3:36 PM Tomasz Urbaszek < > > >> > >>> > tomasz.urbas...@polidea.com> > > >> > >>> > > wrote: > > >> > >>> > > > > >> > >>> > >> Hi all, > > >> > >>> > >> > > >> > >>> > >> I have opened a draft PR with tests run by pytest: > > >> > >>> > >> https://github.com/apache/airflow/pull/6472 > > >> > >>> > >> > > >> > >>> > >> Still, 13 tests are failing. It has increased since friday > > >> where I > > >> > >>> have > > >> > >>> > >> only two failing (the new ones are cassandara and hive > > >> related). > > >> > >>> > >> I share this code to get some inspiration and suggestions > how > > >> can > > >> > we > > >> > >>> > >> approach this subject. Any help is appreciated! > > >> > >>> > >> > > >> > >>> > >> Seeing a number of likes in this thread, should we proceed > > with > > >> > >>> official > > >> > >>> > >> vote for AIP-27 > > >> > >>> > >> < > > >> > >>> > >> > > >> > >>> > > > >> > >>> > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-27+Migrate+to+pytest > > >> > >>> > >>> > > >> > >>> > >> ? > > >> > >>> > >> > > >> > >>> > >> Best wishes, > > >> > >>> > >> Tomek > > >> > >>> > >> > > >> > >>> > >> On Wed, Oct 16, 2019 at 6:26 PM Dan Davydov > > >> > >>> > <ddavy...@twitter.com.invalid> > > >> > >>> > >> wrote: > > >> > >>> > >> > > >> > >>> > >>> +1 > > >> > >>> > >>> > > >> > >>> > >>> On Wed, Oct 16, 2019 at 12:01 PM Christian Lellmann > > >> > >>> > >>> <christian.lellm...@googlemail.com.invalid> wrote: > > >> > >>> > >>> > > >> > >>> > >>>> +1 from my side too. > > >> > >>> > >>>> > > >> > >>> > >>>> Regards, > > >> > >>> > >>>> > > >> > >>> > >>>> Chris > > >> > >>> > >>>> > > >> > >>> > >>>> Driesprong, Fokko <fo...@driesprong.frl> schrieb am Mi., > > 16. > > >> > Okt. > > >> > >>> > >> 2019, > > >> > >>> > >>>> 17:01: > > >> > >>> > >>>> > > >> > >>> > >>>>> +1 > > >> > >>> > >>>>> > > >> > >>> > >>>>> Op ma 14 okt. 2019 om 16:50 schreef Felix Uellendall > > >> > >>> > >>>>> <felue...@pm.me.invalid > > >> > >>> > >>>>>> : > > >> > >>> > >>>>> > > >> > >>> > >>>>>> +1, successfully using pytest for airflow plugins for > > >> almost 2 > > >> > >>> > >> years > > >> > >>> > >>>> now. > > >> > >>> > >>>>>> > > >> > >>> > >>>>>> Felix > > >> > >>> > >>>>>> > > >> > >>> > >>>>>> Sent from ProtonMail Mobile > > >> > >>> > >>>>>> > > >> > >>> > >>>>>> On Mon, Oct 14, 2019 at 16:39, Daniel Imberman < > > >> > >>> > >>>>>> dimberman.opensou...@gmail.com> wrote: > > >> > >>> > >>>>>> > > >> > >>> > >>>>>>> Huge +1 from me on this one! > > >> > >>> > >>>>>>> > > >> > >>> > >>>>>>> On Mon, Oct 14, 2019 at 10:46 AM Bas Harenslak < > > >> > >>> > >>>>>>> basharens...@godatadriven.com> wrote: > > >> > >>> > >>>>>>> > > >> > >>> > >>>>>>>> Big +1 for pytest > > >> > >>> > >>>>>>>> > > >> > >>> > >>>>>>>> Bas > > >> > >>> > >>>>>>>> > > >> > >>> > >>>>>>>>> On 14 Oct 2019, at 15:29, Jarek Potiuk < > > >> > >>> > >>> jarek.pot...@polidea.com> > > >> > >>> > >>>>>> wrote: > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>> Fantastic! Pytest is sooo much better than > nosetests! > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>> On Mon, Oct 14, 2019 at 1:44 PM Ash Berlin-Taylor < > > >> > >>> > >>> a...@apache.org > > >> > >>> > >>>>> > > >> > >>> > >>>>>>>> wrote: > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>>> Awesome, thanks! > > >> > >>> > >>>>>>>>>> > > >> > >>> > >>>>>>>>>> I tried this at the start of the year but we were a > > few > > >> > 100 > > >> > >>> > >>>> failing > > >> > >>> > >>>>>>>> tests > > >> > >>> > >>>>>>>>>> away. > > >> > >>> > >>>>>>>>>> > > >> > >>> > >>>>>>>>>> -ash > > >> > >>> > >>>>>>>>>> > > >> > >>> > >>>>>>>>>>> On 14 Oct 2019, at 11:44, Tomasz Urbaszek < > > >> > >>> > >>>>>> tomasz.urbas...@polidea.com > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>>> wrote: > > >> > >>> > >>>>>>>>>>> > > >> > >>> > >>>>>>>>>>> Hi all, > > >> > >>> > >>>>>>>>>>> > > >> > >>> > >>>>>>>>>>> I've just created AIP-27 Migrate to pytest > > >> > >>> > >>>>>>>>>>> < > > >> > >>> > >>>>>>>>>> > > >> > >>> > >>>>>>>> > > >> > >>> > >>>>>> > > >> > >>> > >>>>> > > >> > >>> > >>>> > > >> > >>> > >>> > > >> > >>> > >> > > >> > >>> > > > >> > >>> > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-27+Migrate+to+pytest > > >> > >>> > >>>>>>>>>>> . > > >> > >>> > >>>>>>>>>>> I think this proposal will improve our test suite > > >> > especially > > >> > >>> > >>>> that > > >> > >>> > >>>>> it > > >> > >>> > >>>>>>>>>> seems > > >> > >>> > >>>>>>>>>>> that we only need to fix 16 tests to obtain green > CI > > >> > build. > > >> > >>> > >>>>>>>>>>> > > >> > >>> > >>>>>>>>>>> Your considerations and feedback are highly > > welcomed! > > >> > >>> > >>>>>>>>>>> > > >> > >>> > >>>>>>>>>>> Bests, > > >> > >>> > >>>>>>>>>>> Tomek > > >> > >>> > >>>>>>>>>> > > >> > >>> > >>>>>>>>>> > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>> -- > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>> Jarek Potiuk > > >> > >>> > >>>>>>>>> Polidea <https://www.polidea.com/> | Principal > > Software > > >> > >>> > >>> Engineer > > >> > >>> > >>>>>>>>> > > >> > >>> > >>>>>>>>> M: +48 660 796 129 <+48660796129> > > >> > >>> > >>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >> > >>> > >>>>>>>> > > >> > >>> > >>>>>>>> > > >> > >>> > >>>>> > > >> > >>> > >>>> > > >> > >>> > >>> > > >> > >>> > >> > > >> > >>> > >> > > >> > >>> > >> -- > > >> > >>> > >> > > >> > >>> > >> Tomasz Urbaszek > > >> > >>> > >> Polidea <https://www.polidea.com/> | Junior Software > > Engineer > > >> > >>> > >> > > >> > >>> > >> M: +48 505 628 493 <+48505628493> > > >> > >>> > >> E: tomasz.urbas...@polidea.com < > tomasz.urbasz...@polidea.com > > > > > >> > >>> > >> > > >> > >>> > >> Unique Tech > > >> > >>> > >> Check out our projects! <https://www.polidea.com/our-work> > > >> > >>> > >> > > >> > >>> > > > > >> > >>> > > > > >> > >>> > > -- > > >> > >>> > > > > >> > >>> > > Jarek Potiuk > > >> > >>> > > Polidea <https://www.polidea.com/> | Principal Software > > >> Engineer > > >> > >>> > > > > >> > >>> > > M: +48 660 796 129 <+48660796129> > > >> > >>> > > [image: Polidea] <https://www.polidea.com/> > > >> > >>> > > > >> > >>> > > > >> > >>> > > >> > >> > > >> > > >> > > >> > > >> -- > > >> > > >> Chao-Han Tsai > > >> > > > > > > > > > -- > > > > > > Tomasz Urbaszek > > > Polidea <https://www.polidea.com/> | Junior Software Engineer > > > > > > M: +48 505 628 493 <+48505628493> > > > E: tomasz.urbas...@polidea.com <tomasz.urbasz...@polidea.com> > > > > > > Unique Tech > > > Check out our projects! <https://www.polidea.com/our-work> > > > > > > > > > -- > > > > Tomasz Urbaszek > > Polidea <https://www.polidea.com/> | Junior Software Engineer > > > > M: +48 505 628 493 <+48505628493> > > E: tomasz.urbas...@polidea.com <tomasz.urbasz...@polidea.com> > > > > Unique Tech > > Check out our projects! <https://www.polidea.com/our-work> > > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> >
